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Preface 



The creation of intelligent robots is surely one of the most exciting and chal- 
lenging goals of Artificial Intelligence. A robot is, first of all, nothing but an 
inanimate machine with motors and sensors. In order to bring life to it, the 
machine needs to be programmed so as to make active use of its hardware com- 
ponents. This turns a machine into an autonomous robot. Since about the mid 
nineties of the past century, robot programming has made impressive progress. 
State-of-the-art robots are able to orient themselves and move around freely in 
indoor environments or negotiate difficult outdoor terrains, they can use stereo 
vision to recognize objects, and they are capable of simple object manipulation 
with the help of artificial extremities. 

At a time where robots perform these tasks more and more reliably, we are 
ready to pursue the next big step, which is to turn autonomous machines into 
reasoning robots. A reasoning robot exhibits higher cognitive capabilities like 
following complex and long-term strategies, making rational decisions on a high 
level, drawing logical conclusions from sensor information acquired over time, 
devising suitable plans, and reacting sensibly in unexpected situations. All of 
these capabilities are characteristics of human-like intelligence and ultimately 
distinguish truly intelligent robots from mere autonomous machines. 

What are Robotic Agents? 

A fundamental paradigm of Artificial Intelligence says that higher intelligence is 
grounded in a mental representation of the world and that intelligent behavior 
is the result of correct reasoning with this representation. A robotic agent is 
a high-level control program for a robot — or, for that matter, for a proactive 
software agent — in which such mental models are employed to draw logical con- 
clusions about the world. Intelligent robots need this technique for a variety of 
purposes: 

• Reasoning about the current state. 

What follows from the current sensor input in the context of the 
world model ? 

• Reasoning about action preconditions. 

Which actions are currently possible ? 
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• Reasoning about effects. 

What holds after an action has been taken ? 

• Planning. 

What needs to he done in order to achieve a given goal ? 

• Intelligent troubleshooting. 

What went wrong and why, and what could be done to recover ? 

Research on how to design an automatic system for reasoning about actions 
has a long history in Artificial Intelligence. The earliest formal model for the 
ability of humans to solve problems by reasoning has been the so-called situa- 
tion calculus, whose roots go back to the early sixties. In the late sixties this 
model has been used to build an automatic problem solver. However, this first 
implementation did not scale up beyond domains with a small state space and 
just a few actions because it suffered from what soon became a classic in Artifi- 
cial Intelligence, the so-called frame problem. In a nutshell, the challenge is to 
describe knowledge of effects of actions in a succinct way so that an automatic 
system can efficiently update an internal world model upon the performance 
of an action. The frame problem has haunted researchers for many years, and 
only in the early nineties the first satisfactory solutions have emerged. These 
formal models for reasoning about actions are now being developed into actual 
programming languages and systems for the design of robotic agents. 

One successful approach to the frame problem is provided by a formalism 
known as fluent calculus. This book is concerned with this model of rational 
thought as a way to sepcify mental models of dynamic worlds and to reason 
about actions on the basis of these models. Very recently the calculus has 
evolved into the programming method and system FLUX, which supports the 
problem-driven, top-down design of robotic agents with the cognitive capabilities 
of reasoning, planning, and intelligent troubleshooting. 



Why Fluent Calculus? 

Fluent calculus originates in the classical situation calculus. It provides the for- 
mal underpinnings for an effective and computationally efficient solution to the 
fundamental frame problem. To this end, fluent calculus extends situation cal- 
culus by the basic notion of a state, which allows to define effects very naturally 
in terms of how an action changes the state of the world. Based on classical 
predicate logic, fluent calculus is a very versatile formalism, which captures a 
variety of phenomena that are crucial for robotic agents, such as incomplete 
knowledge, nondeterministic actions, imprecise sensors and effectors, and indi- 
rect effects of actions as well as unexpected failures when an action is performed 
in the real, unpredicatble world. 
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Why FLUX? 

FLUX is a Prolog-based method for programming robotic agents based on the 
expressive action theory of fluent calculus. It comprises a way of encoding 
incomplete world models along with a technique for updating these models 
according to a declarative specification of the acting and sensing capabilities of a 
robot. Using a powerful constraint solver, a generic base system provides general 
reasoning facilities, so that the agent programmer can focus on specifying the 
application domain and designing the intended high-level behavior. Allowing 
for concise programs and supporting modularity, FLUX is eminently suitable 
for programming complex strategies for robotic agents. Thanks to a restricted 
expressiveness and a sound but incomplete inference engine, the system exhibits 
excellent computational behavior. In particular, it scales up well to long-term 
control thanks to the underlying principle of “progression,” which means to 
continually update a (possibly incomplete) world model upon the performance 
of an action. Appealing to a declarative programming style, FLUX programs 
are easy to write, understand, and maintain. The book includes the details of 
the base system written in Prolog, so that it can be easily adapted and extended 
according to one’s own needs. 



Further Aspects 

This introductory motivation would not be complete without the admission 
that the current state-of-the-art in research on reasoning robots still poses fun- 
damentally unsolved problems. Maybe the most crucial issue is the interaction 
between cognitive and low-level control of a robot, in particular the question 
about the origin of the symbols, that is, the names for individuals and cate- 
gories that are being used by a robotic agent. In this book we take a pragmatic, 
top-down approach, which requires the designer of a system to predefine the 
grounding of the symbols in the perceptual data coming from the sensors of a 
robot. Ultimately, a truly intelligent robot must be able to handle this symbol 
grounding problem by itself in a more flexible and adaptive manner. Another 
important issue is the lack of self-awareness and true autonomy. The reasoning 
robots considered in this book are able to follow complex strategies and they 
are capable of devising and executing their own plans. Still both the intended 
behavior and the boundaries for the plans are determined by the programmer. 
Ultimately, a truly self-governing robot must be able to reflect and adapt its 
behavior when facing new and unforeseen situations. 



What’s in this Book? 

This book provides an in-depth and uniform treatment of fluent calculus and 
FLUX as a mathematical model and programming method for robotic agents. 
As theory and system unfold, the agents will become capable of dealing with in- 
complete world models, which require them to act cautiously under uncertainty; 
they will be able to explore unknown environments by logically reasoning about 
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sensor inputs; they will plan ahead some of their actions and react sensibly to 
action failure. 

The book starts in Chapter 1 with an introduction to the axiomatic formal- 
ism of fluent calculus as a method both for specifying internal models of dynamic 
environments and for updating these models upon the performance of actions. 
Based on this theory, the logic programming system FLUX is introduced in 
Chapter 2 as a method for writing simple robotic agents. The first two chapters 
are concerned with the special case of agents having complete knowledge of all 
relevant properties of their environment. In Chapters 3 and 4, theory and system 
are generalized to the design of intelligent agents with incomplete knowledge 
and which therefore have to act under uncertainty. Programming these agents 
relies on a formal account of knowledge given in Chapter 5, by which conditions 
in programs are evaluated on the basis of what an agent knows rather than 
what actually holds. Chapter 6 is devoted to planning as a cognitive capabil- 
ity that greatly enhances flexibility and autonomy of agents by allowing them 
to mentally entertain the effect of different action sequences before choosing 
one that is appropriate under the current circumstances. Chapters 7 and 8 are 
both concerned with the problem of uncertainty due to the fact that effects of 
actions are sometimes unpredictable, that state properties in dynamic environ- 
ments may undergo changes unnoticed by an agent, and that the seonsors and 
effectors of robots are always imprecise to a certain degree. Chapter 9 deals with 
a challenge known as the ramification problem. It arises in complex environ- 
ments where actions may cause chains of indirect effects, which the agent needs 
to take into account when maintaining its internal world model. Chapter 10 is 
devoted to an equally important challenge known as the qualification prob- 
lem. It arises in real-world applications where the executability of an action 
can never be predicted with absolute certainty since unexpected circumstances, 
albeit unlikely, may at any time prevent the successful performance of an action. 
The solution to this problem enables agents to react to unexpected failures by 
generating possible explanations and revising their intended course of actions 
accordingly. Finally, Chapter 11 describes a system architecture in which 
robotic agents are connected to a low-level, reactive control layer of a robot. 
The appendix contains a short user manual for the FLUX system. 

The only prerequisite for understanding the material in this book is basic 
knowledge of standard first-order logic. Some experience with programming 
in Prolog might also be helpful. Sections marked with * contain technical 
details that may be skipped at first reading or if the reader’s main interest is in 
programming. The book has its own webpage at 

WWW .fiuxagent. org 

where the FLUX system and all example programs in this book are available 
for download. 
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Chapter 1 



Special Fluent Calculus 



Imagine a robot sitting in a hallway with a number of offices in a row. The robot 
is an automatic post boy, whose task is to pick up and deliver in-house mail 
exchanged among the offices. To make life easier for the robot, the documents 
that are being sent from one office to another are all together put into one 
standardized delivery package. The robot is equipped with several slots, a kind 
of mail bag, each of which can be filled with one such package. Initially, however, 
there may be many more delivery requests than the robot can possibly carry 
out in one go, given its limited capacity. Figure 1.1 depicts a sample scenario in 
an environment with six offices and a robot with three mail bags. A total of 21 
packages need to be delivered. The question is how to write a control program 
which sends the robot up and down the hallway and tells it where to collect and 
drop packages so that in the end all requests have been satisfied. 

Actually, it is not too difficult to come up with a simple, albeit not necessarily 
most efficient, strategy for the “mailbot:” Whenever it finds itself at some office 
for which it carries one or more packages, then these are delivered. Conversely, 
if the robot happens to be at some place where items are still waiting to be 
collected, then it arbitrarily picks up as many as possible, that is, until all 
bags are filled. If no more packages can be dropped nor collected at its current 
location, the robot makes an arbitrary decision to move either up or down the 
hallway towards some office for which it has mail or where mail is still waiting 
to be picked up. 

Following this strategy, our robot in Figure 1.1, for example, chooses to pick 
up all three packages in the first room and then to move up to room number 2, 
where it can deliver one of them. Thereafter, it may select the package which 
is addressed from room 2 to 3, and to move up again. Arriving at room 
number 3, the robot can drop both the package coming from room 1 and the 
one from room 2. This leaves the robot with two empty bags, which it fills 
again, and so on. At the end of the day, there will be no unanswered requests 
left and the robot will have emptied again all its mail bags. 

The simple strategy for the mailbot is schematically depicted in Figure 1.2. 
Using a pseudo-language, the algorithm is implemented by the following pro- 
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1 2 3 4 5 6 




Figure 1.1: The initial state of a sample mail delivery problem, with a total 
of 21 requests. 



gram: 

loop 

if possible to drop a package 
then do it 

else if possible to pick up a package 
then do it 

else if can pick up or drop a package up (resp. down) the hallway 
then go up (resp. down) 

else stop 
end loop 

Obviously, this program requires our mailbot to evaluate conditions which de- 
pend on the current state of affairs. For in order to decide on its next action, it 
always needs to know the current contents of its mail bags, the requests that are 
still open, and its current location. Since these properties constantly change as 
the program proceeds and not all of them can be directly sensed, the robot has 
to keep track of what it does as it moves along. For this purpose, our mailbot 
is going to maintain an internal model of the environment, which throughout 
the execution of the program conveys the necessary information about the cur- 
rent location of all packages that have not yet been delivered. The model needs 
to be updated after each action in accordance with the effects of the action. 
With regard to the scenario in Figure 1.1, for instance, if the robot starts with 
putting a particular package into one of its mail bags, this will be recorded as a 
modification of the model, namely, by removing the corresponding request and 
adding the information into which mail bag the package in question has been 
put. Likewise, whenever a package is actually taken out of a mail bag, this has 
to be done mentally as well, in order for the robot to know that this bag can be 
filled again. 

Fluent calculus lays the theoretical foundations for programming robotic 
agents like our mailbot that base their actions on internal representations of the 
state of their environment. The theory provides means to encode internal models 
for agents and to describe actions and their effects. As the name suggests, the 
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begin 




end 



Figure 1.2: A high-level algorithm for mail delivery. 



calculus also tells us how to calculate the update of a model whenever an 
action is performed. In this first chapter, we will focus on a special variant of 
fluent calculus which is suitable for agents that always know all relevant facts of 
their environment. The mail delivery robot falls into this category if we assume 
that all requests are given initially and do not change during program execution. 

The chapter is divided into three sections, which introduce the three basic 
components of a problem-oriented description of a robot domain: 

1.1. the states of the environment; 

1.2. the actions of the robot; 

1.3. the effects of the actions on the environment. 

1.1 Fluents and States 

Typically, the state space tends to be quite large even in simple robot environ- 
ments. For our mail delivery scenario with just six offices and three mail bags, 
for example, we can distinguish more than 2-10^^ states, considering all possible 



4 



CHAPTER 1. SPECIAL FLUENT CALCULUS 



combinations of requests, fillings of the bags, and locations of the robot. This 
number increases exponentially with the size of the office floor or the capacity 
of the robot. In many applications there is even no upper bound at all for the 
state space. It is therefore advisable for agents to adopt a principle known as 
“logical atomism,” which means to break down states into atomic properties. 
The state components of the mail delivery world, for example, are the current 
requests, the contents of the mail bags, and the location of the robot. 

By convention, atomic properties of states are called fluents, thereby hinting 
at their fleeting nature as time goes by. Maintaining a model of the environment 
during program execution is all about how the various fluents change due to the 
performance of actions. Fluent calculus is named after these state components 
and provides means to calculate the changes they undergo when the agent acts. 

Generally speaking, fluent calculus is a language that uses standard predicate 
logic with a few customary conventions regarding sorts and equality. 



Sorts can be used in logic to define the range of the arguments of predicates 
and functions. For example, a function Pickup might be specified as taking 
two arguments of some sort called N (for: natural numbers), yielding a term 
of another sort called ACTION; similarly, predicate Pass (for: possible) may 
be specified as taking two arguments of sort ACTION and another sort called 
STATE. Sorts need not be disjoint; e.g., natural numbers N are a sub-sort of 
integers Z, which in turn are contained in the real numbers K. Variables in 
formulas are sorted, too, and may be substituted by terms of the right sort 
only. For example, in the formula (3x, z) Poss{Pickup{2, x)^ z) , the variable 
X must be of sort IN and z of sort state. For conventional sorts like the 
natural numbers we use the standard arithemtic operations with their usual 
interpretation. We also use the equality predicate “=” (defined for all pairs 
of elements of the same sort), which is assumed to be interpreted as the identity 
relation. 



For modeling states of an environment, fluent calculus uses two basic sorts, 
namely, for fluents and for states. Intuitively, any collection of fluents makes a 
state in which exactly these fluents hold. Formally, each fluent itself constitutes 
a (singleton) state, and there is a binary connection function that allows to form 
a new state by taking the union of two states. Furthermore, a special constant 
denotes the empty state. 

Deflnition 1.1 A triple (IF, o,0) is a fluent calculus state signature if 

• T finite, non-empty set of function symbols into sort fluent 

• O : STATE X STATE i— > STATE 



0 : STATE. 
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A fluent is a term of sort fluent. A state is a term of sort state, with 
FLUENT being a sub-sort of state. □ 

For the sake of better readability, the binary connection function will be written 
in infix notation, as in zi o Z 2 . Throughout the book, fluent variables will be 
written by the letters f ox g and state variables by the letter z, all possibly 
with sub- or superscripts. Conventions like this will allow us to refrain, in most 
cases, from explicitly stating the sorts of the variables in formulas. 

Example 1 Numbering both the offices and the robot’s mail bags, a state 
in the mail delivery world will be described with the help of the following four 
fluents: 

At\ N 1-^ FLUENT At{r) = robot is at room r 

Empty : N ^ fluent Empty {b) = mail bag b is empty 

Carries : N x N fluent Carries {b, r) = bag b has a package for r 

Request : N x N fluent Request {r 1 ^x 2 ) = request from r\ to T2 

Using the connection function, the initial state depicted in Figure 1.1 can then 
be formalized by this term: 

At{l) o (^Empty{l) o (^Empty(2) o (^Empty{5) o , , 

{Request(l, 2) o (Request(l,S) o ... o Requested, A) .. .))))) 

□ 

Defining fluents is a matter of design. The fluent Empty, for example, is some- 
what redundant because emptiness of a mail bag could also be indirectly ex- 
pressed as not carrying anything in this bag, using the fluent Carries. On the 
other hand, the additional fluent both makes specifications more readable and 
allows to check conditions more efficiently. 

In agent programs, conditions which refer to the state of the outside world 
are based on the notion of fluents to hold in states. For example, it is impossible 
for the mailbot to put some package into mail bag b if fluent Emptyib) does 
not hold in the current state. Likewise, the contents of some bag b can be 
delivered just in case Carries(b,r) and At{r) hold for the same r, that is, the 
robot needs to be at the very room to which the package is addressed. The 
property of a fluent to hold in a state is formally expressed by an equational 
formula which says that the state needs to be decomposable into the fluent plus 
some arbitrary other sub-state. Since this notion is almost ubiquitous in fluent 
calculus formulas, it is abbreviated by a macro: 

Holds{f : fluent, z : state) = {3z) z = f o z (1.2) 

Macros constitute a kind of surface language which helps making logical axioms 
both more readable and handier. Whenever a macro occurs in a formula, it has 
to be taken as a mere abbreviation for the actual sub-formula it stands for. As 
we unfold the theory of fluent calculus, further macros will be introduced for 
the benefit of the reader. 
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Some words on the logic notation used in this book. Formulas are built 
up from atoms and the standard logical connectives, stated in order of de- 
creasing priority: V (universal quantification), 3 (existential quantifi- 
cation), ^ (negation) , A (conjunction) , V (disjunction) (impli- 
cation), = (equivalence), and T and _L (true and false, respectively). 
Throughout the book, both predicate and function symbols start with a cap- 
ital letter whereas variables are in lower case, sometimes with sub- or super- 
scripts. Sequences of terms like xi,...,Xn are often abbreviated as x. Vari- 
ables outside the range of the quantifiers in a formula are implicitly assumed 
universally quantified, unless stated otherwise. 



In order to capture the intuition of identifying states with the fluents that 
hold, the special connection function of fluent calculus obeys certain properties 
which closely resemble those of the union operation for sets. 

Definition 1.2 The foundational axioms Esstate of special fluent cal- 
culus are,^ 

1. Associativity and commutativity, 

(zi O Z2) O Z3 = Zi O {Z 2 O Z3) 

Z \0 Z 2 = Z20 Zi 



2 . 

3. 

4. 

5. 



Empty state axiom, 

Irreducibility, 

Holds (f,g) D f = g 



Decomposition, 

Holds{f, zi o Z2) D Holds{f,zi) V Holds{f,Z2) 

State equality, 

(V/) {Holds{f,zi) = Holds{f,Z2)) D zi = Z2 



□ 

Associativity and commutativity imply that it does not matter in which order 
the fluents occur in a state term. The axiom of associativity permits us to 
omit parenthesis on the level of “o” from now on. The empty state axiom says 
that no fluent is contained in the special state 0. The axioms of decomposition 

^ The first “s” in J^sstate indicates that these axioms characterize the special version of 
fluent calculus. 
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and irreducibility imply that compound states can be decomposed into single 
fluents and not further. The very last foundational axiom says that states are 
indistinguishable if they contain the same fluents. 

The foundational axioms of special fluent calculus entail, for example, 

(3b) Holds(Empty(b), Z) 

(where Z shall be the state in (1.1)) since Esstate H Z = Empty(b) o z' . 

On the other hand, Egstate does not entail 

(36, x) (Holds(At(x), Z) A Holds(Carries(b, x), Z)) 

Hence, in the initial state the robot can put something into one of its mail bags 
while there is nothing that can be delivered. 

The foundational axioms together are consistent, as can be easily shown by 
constructing a standard model which interprets states as sets of fluents. 

Proposition 1.3 E^^tate is consistent. 

Proof: Let t be an interpretation whose domain includes all sets of ground 

fluents and where the mapping is defined by 

1 . 0 ‘ = {}, 

2- /'' = {/} for each ground fluent /, 

3. (zi O Z2)‘ = U Z2- 

Recall that macro Holds(f, z) stands for (3z') z = f o z' , which is true under l 
just in case there exists a set Z' such that Z = {F} U Z' , where Z = z'' and 
F = f'' . Interpretation i is then easily verified to be a model for Esstate' 

1. Set union is an associative-commutative operation. 

2. There are no sets {F} and Z such that {} = {F} U Z. 

3. {G} = {F} U Z implies F = G. 

4. If {F} U Z = Zi U Z 2 , then F ^ Zi or F £ Z 2 ; hence, {F} U Z' = Zi 
or {F} U Z' = Z 2 for some set Z' . 

5. For any F, suppose that there exists some Z with {F} \J Z = Z\ if and 

only if there also exists some Z' such that {F} U Z' = Z 2 . Then F G Zi 
iff F G Z 2 ] hence, Z\ = Z 2 . ■ 

The foundational axioms are also mutually independent, that is, there is no 
redundancy in E^state (see Exercise 1.2). 

The standard model raises the question why we cannot use sets in the first 
place instead of introducing and axiomatically characterizing a special connec- 
tion function. The reason is that the approach taken in fluent calculus consti- 
tutes an extensional definition of sets and set operations, as will be shown soon. 
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The extensional view is extremely useful when it comes to logical reasoning with 
incomplete specifications of states,^ in which case the foundational axioms give 
rise to short logical derivations. This is in contrast to the standard character- 
ization of sets, which is intensional and, for example, provides no means for 
quickly inferring the result of removing an element from an incompletely speci- 
fied set. This issue will be raised again in Chapter 3 when introducing general 
fluent calculus, which deals with incomplete state information. 

In special fluent calculus, agents are assumed to always know exactly which 
fluents hold in their environment. Speaking formally, we are particularly inter- 
ested in states consisting of an explicit enumeration of fluents. 

Definition 1.4 A finite state r is a term fio...of^ such that each fi 
(1 < * < n) is a fluent (n > 0). If n = 0, then t is 0. A gronnd state is a 
finite state without variables. □ 

State (1.1), for example, is ground while the term mentioned in footnote 2 is, 
by definition, not a finite state due to the occurrence of state variable z (in 
which infinitely many fluents may hold). 

It is easy to see that finite states tell us precisely which fluents hold in them. 

Proposition 1.5 Let r = /i o . . . o /„ be a finite state (n > 0), then 

n 

^sstate h Holds{f,T) = \f {f, = f) 
i=l 

Proof: Let / be a fluent. If Vr=i(/j = /)’ n > 1 and (3z) t = f o z 

according to associativity and commutativity; hence, Holds{f,T) according to 
macro definition (1.2). 

The converse is proved by induction on n. If n = 0, then the empty state 
axiom implies Holds{f,r) = T. Suppose the claim holds for n, and consider 
Holds{f,T o fn+i). Decomposition implies Holds{f,T) \/ Holds{f, fn+i) ■ The 
induction hypothesis and irreducibility imply = f)^ f = fn+i- ■ 

Agents maintain a state description as their internal world model, against 
which conditional program statements are verified. Since their actions affect the 
outside world, agents need to constantly update their model as they move along. 
In general, actions cause some fluents to become false and others to become true. 
Whenever, say, our mailbot in room ri grabs and puts into bag b a package for 
room T 2 , then the corresponding fluent Carries{b,r 2 ) is true afterwards. The 
fluent Empty (b), on the other hand, ceases to hold, and so should the fluent 
Request{ri,r 2 ) in order that the robot is not tempted to pick up the package 
again. Addition and removal of fluents from states are therefore two important 
operations on states. The addition of a fluent / to a state z is easily specified 
by zo / . Axiomatizing the subtraction of fluents, on the other hand, is slightly 
more complicated. 

^ An arbitrary example of an incomplete state is At[x) o Carries{2, x) o z, which says that 
the robot carries in its second bag a package for the office of its current location. It is not 
said which office we are talking about, nor does the term indicate what else holds. 



1.1. FL UENTS AND STATES 



9 



Below, we introduce the macro equation zi — (/i o . . . o /„) = Z 2 with the 
intended meaning that state Z 2 is state Zi minus the fluents /i ,■■■,/« • The 
crucial item in the inductive deflnition is the second one, by which removal of 
a single fluent is axiomatized as a case distinction, depending on whether the 
fluent happens to be true or not. The first and third item define, respectively, 
the special case of removing the empty state and the generalization to removal 
of several fluents: 

0 de£ 

= Z2 = Z2 = zi; 

• Zi - f = Z2 = {Z2 = Zi V Z20 f = Zi) A ^Holds{f, Z2); 

• Zi-{fi0f2 0...0 /„) = Z2 = ( 3 z) (zi - /i = z A z - (/2 o . . . O /„) = Z2) 

where n > 2. 

The intuition behind the second item is easy to grasp with the equivalent, 
but less compact, formulation of the right hand side as the conjunction of 
-^Holds{f,z\)Z)Z 2 = zi and Holds{f,zi)Z)Z 2 of = zi/\^Holds{f,Z 2 )- 

The following proposition shows that this extensional deflnition of removal 
of finitely many elements is correct wrt. the standard intensional deflnition of 
set difference. 

Proposition 1.6 Let t = fi o ... o /„ be a finite state (n >0). Then 

Esstate h zi-t = Z2 A [Holds{f , Z2) = Holds {f , Zi) A ^ Holds{f , t)] 

Proof: Suppose zi — fi o ... o = Z2. The proof is by induction on n. 

If n = 0, then z\ = Z2', hence, Holds{f, Z2) = Holds{f , z\) according to the 
deflnition of Holds. With the empty state axiom it follows that Holds {f, Z2) = 
Holds{f, Zi) A ~^Holds{f, 0). 

Suppose the claim holds for n, and consider Z\ — (/n+i o r) = Z 2 . Macro 
expansion implies that there exists some z such that z\ — fn+i = z and 
z — T = Z2, that is. 



Suppose Holds{f,Z 2 ), then equation (1.5) and the induction hypothesis im- 
ply Holds{f,z) A^Holds{f,T). From (1.3) it follows that Holds{f,z\). More- 
over, (1.4) and Holds{f,z) imply / f fn+i', hence, ~^Holds{f, fn+i ° t) ac- 
cording to the axioms of decomposition and irreducibility. 

Conversely, suppose Holds{f, zi) A ^Holds{f, fn+i o r), then decomposi- 
tion implies ~^Holds{f, f„+i) and ~^Holds{f,T). From (1.3), decomposition 
and ~^Holds{f, fn+i) it follows that lfoWs(/, z); hence, (1.5) and ^Holds{f,T) 
along with the induction hypothesis imply Holds {f, Z2). ■ 

While the intensional, “piece-wise” deflnition of set difference is both more com- 
mon and more general (as it is not restricted to removal of finite sets), the 



z = zi V z o /„+i = Zi 
-nHolds{f„+i,z) 

Z-T = Z2 



(1.3) 

(1.4) 

(1.5) 
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extensional definition is better suited for actually calculating the state term 
which results from the removal of fluents, in particular if the original state is 
incompletely specified. 

On top of the macro for specifying negative effects, the macro equation 
Z 2 = (zi — '&~) + abbreviates an axiom which stipulates that state Z 2 is 
state zi minus the fluents in D plus the fluents in ■d’’'. Both D and i?"*" 
are finite states: 



2^2 = ^1 - /l O • ■ • O /m + ffl o • • ■ O ffn = (3z) { Zi - fi O . . . O = Z 

A Z2 = zogio ...o g„) 

State update is thus defined by an update equation, which an agent needs to 
solve in order to know what holds after it has performed an action. The following 
computation rule is very useful to this end. It says that equal fluents on both 
sides of an equation can be cancelled out under certain conditions. 

Proposition 1.7 (Cancellation Law) 

^sstate h -^Holds{f, Zi) A ~^Holds{f, Z 2 ) D [ziO f = Z 20 f D Zi = Z 2 ] 

Proof: Suppose that ~^Holds{f, zi), ~^Holds{f , Z 2 ) and zi o f = Z 2 o f . 

We show that Holds{f , zi) = Holds{f , Z 2 ), which proves the claim according 
to the axiom of state equality. If Holds{f',Zi), then Holds{f , Zi o /); hence, 
Holds{f',Z 2 °f) since Ziof = z 2 °f. From Holds{f',Zi) and ^Holds{f,Zi) 
it follows that f This and Holds{f , Z 2 o f) implies Holds{f',Z 2 ) due to 
decomposition and irreducibility. The converse is similar. ■ 

In case of complete states, solving the state equations is of course fairly 
trivial as one can just apply the corresponding set operations. As an example, 
let Zi be the initial state (1.1) of our mail delivery problem, and consider the 
following equation: 

Z 2 = Zi — Empty{3) o Request(l,2) + Carries{3,2) 

This update from state Zi to Z2 formalizes the effect of putting the package 
from room 1 to 2 into mail bag 3. Macro expansion implies the existence of 
z',z” such that 



z' = Zi — Empty {3) 
z!' = z — Request {1,2) 

Z 2 = z" o Carries {3, 2) 

Since Empty {3) holds in Zi, the first equation implies z' o Empty {3) = Zi 
and ^ Holds {Empty {3), z') . The latter allows to apply the cancellation law 
after inserting (1.1) for Zi, which yields 

z' = At{l) o Empty{l) o Empty{2) o 

Request{l, 2) o Request{l, 3) o . . . o Request{6, 4) 
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In the same fashion we conclude 

z" = At{T) o Empty{l) o Empty{2) o Request{l, 3) o . . . o Request{6, 4) 
Finally, 



Z 2 = ^^(1) o Empty{l) o Empty{2) o 

Request{l, 3) o . . . o Request{(S, 4) o Carries{3, 2) 

where mail bag 3 is no longer empty but contains a package for office 2, and 
where the request for a delivery from 1 to 2 is no longer present. 

Actually, the cancellation law applies in the example just given only if the flu- 
ents Empty (1) and Request{l,2) do not equal any other fluent in Zi. Axioms 
defining inequality of terms which use different function symbols or arguments 
are very common in fluent calculus. Informally speaking, they say that different 
names refer to different objects. 

Definition 1.8 A unique- name axiom is a formula of the form 

n— 1 n n 

f\ f\hi{x) hj{y) A f\[hi{x) = hi{y) Z) X = y] 

i—1 i—1 

abbreviated UNA[hi , . . . , h,,] . □ 

The first part of a unique-name axiom stipulates that terms with different lead- 
ing function symbol are unequal; e.g., At{ri) ^ Carries{b,r 2 ) . The second 
part implicitly says that terms are unequal which start with the same func- 
tion symbol but whose arguments differ; e.g., Empty(l) =/^ Empty(A) since 
1 7^ 3 under the standard interpretation of natural numbers and equality. 
Given uniqueness of names for all fluent functions in the mail delivery world, 
i.e., UNA[At, Empty , Carries, Request], the conditions of the cancellation law 
are guaranteed to hold in the example given before. 

1.2 Actions and Situations 

In agent programming, actions mean the interactions of the agent with its en- 
vironment. Actions may change the outside world, e.g., when a robot picks up 
a package or a software agent orders a product over the Internet . Other actions 
only change the status of the physical agent itself, e.g., when a robot moves to 
a new position. Finally, actions may just provide the agent with information 
about the environment, e.g., when a robot senses whether a door is open or 
a software agent compares prices at different online stores. While a single ac- 
tion can be a very complex behavior on the level of the physical agent, actions 
are taken as elementary entities on the level of agent programs and in fluent 
calculus. 

Agent programming is all about triggering the agent to do the right action at 
the right time. Every agent program should produce a sequence of actions to be 
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Figure 1.3: A tree of situations. 



executed by the agent. When talking about the different stages of an actual run 
of a program, we refer to the sequence of actions that have been performed up 
to a point as a situation. The special constant So denotes the initial situation, 
in which the agent has not yet done any interaction with the outside world. The 
constructor Do{a, s) then maps an action a and a situation s to the situation 
after the performance of the action. Hence, action sequences are nested terms 
of the form Do(an, ■ ■ ■ , Do(ai, Sq) . . .). The situations can be visualized as the 
nodes of a tree rooted in 5*0; see Figure 1.3. Each branch in this tree is a 
potential run of a program for the agent. 

Most conditions in agent programs are evaluated against the state which the 
environment is in at the time the program statement is executed. The stan- 
dard function State{s) is used to denote these states at the different stages of 
a program run, i.e., at the different situations s. The initial situation in our 
mail delivery scenario, for instance, is characterized by the equational axiom 
State (Sq) = Z where Z is the term of (1.1). A successful program for the mail- 
bot should then generate a sequence of actions Sn = Do{an, ■ ■ ■ , Do{a\, Sq) . . .) 
such that, for some r, 

State(Sn) = At{r) o Empty{l) o Empty{2) o Empty{3) 

On the formal side, there are special sorts for both actions and situations. 
Completing the signature, the standard predicate Poss{a,z) is used to specify 
the conditions on state z for action a to be executable by the agent. 

Definition 1.9 A tuple 5U {A, Sq, Do, State, Poss) is a fiuent calculus 
signature if 



• S state signature 
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Figure 1.4: The mail delivery robot of Figure 1.1 after having performed the 
action sequence of situation term (1.6). 



• A finite, non-empty set of function symbols into sort action 

• So : SIT 

• Do : ACTION X SIT 1-^ SIT 

• State : SIT state 

• Pass : ACTION X state. 

An action is a term of sort ACTION. A situation is a term of sort sit. □ 

Throughout the book, action variables will be written by the letter a and 
situation variables by the letter s, all possibly with sub- or superscripts. 

Example 1 (continued) The actions of the mailbot shall be denoted by the 
following three function symbols: 

Pickup : N X N I— > ACTION Pickup{b, r) = put in b package for r 

Deliver: N ACTION Deliverih) = deliver contents of bag b 

Go: {Up, Down} i— > ACTION Go{d) = move one office up or down 

Applied to the scenario of Figure 1.1, a successful control program may reach 
the following situation after nine steps, where the robot has already carried out 
three of the 21 delivery tasks (cf. Figure 1.4): 

D 0 {Deliver (2), Do{Deliver{l), Do{Go{Up), 

Do{Pickup{l, “i), Do{D diver {!), Do{Go{ Up), (1.6) 

Do{Pickup{S, 5), Do{Pickup{2, 3), Do{Pickup{l, 2), S'o))))))))) 

On the other hand, there are many situations which a program should never 
generate as they describe action sequences which are impossible to execute. 
Examples are Do{Go{Down), Sq) or Do{Deliver{3), Do{Pickup{l, 2), Sq)) , as- 
suming that the robot starts in state (1.1). □ 
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For notational convenience, we will sometimes write situation terms using 
lists of actions according to the following definition: 

Do{[ai,. . . ,an],s) = Do{an, ■ ■ ■ ,Do{ai,s) . . .) (n > 0) (1.7) 

The situation in (1.6), for example, can thus be abbreviated as 

Do{[Pickup{l, 2), Pickup{2, 3), . . . , Deliver{2)], So) 

The concept of a situation allows us to talk about the state of an environment 
at different times. In particular, we can define what it means for a fluent to 
hold at a certain stage of a program run: 

Holds(f,s) = Holds{f, State(s)) (1-8) 

Based on this definition, one can formulate arbitrary logical assertions about 
the state of the world in a situation. 

Definition 1.10 A state formula A(z) is a first-order formula with free 
state variable z and without any occurrence of states other than in expres- 
sions of the form Holds {f , z), and without actions or situations. If A(z) is a 
state formula and s a situation variable, then A{z){z/ State{s)} is a situation 
formula, written A(s). □ 

For instance, {3r) {Holds {At (r), z) A r > 4) is a state formula, whereas the 
formula {3b) Holds {Empty {b), z o Empty {!)) is not. Likewise, the implication 
{yb) {Holds{Empty{b), s) D ^{3r) Holds{Carries{b,r),s)) is a situation formula, 
whereas {3r) Holds{At{r + 1), Do{Go{Up),s)) is not. 

Situation formulas provide an elegant way of distinguishing those combina- 
tions of fluents which describe a physical impossibility. Usually there are many 
formal states which cannot possibly occur in the real world. The mailbot, for 
example, can never be at more than one location at a time, nor can it ever carry 
a package in a bag which is empty. Hence, the syntactically perfectly acceptable 
state At{l) o At{A) o Empty{2) o Carries{2,3) represents a state of affairs that 
will actually never happen. So-called domain constraints are formulas that need 
to be satisfied in order for a state to possibly arise in reality. 

Definition 1.11 A domain constraint is a formula (Vs)A(s) where A(s) 
is a situation formula. A state z satisfies a set of domain constraints 
wrt. some auxiliary axioms Eaux iff U Eaux U Esstate U {State{s) = z} is 
consistent. □ 

Auxiliary axioms are formulas such as the underlying unique-name axioms 
which describe static properties of the domain. 

Example 1 (continued) The physical reality of the mail delivery world is 
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characterized by the following domain constraints: 

( 3 r) {Holds{At{r), s) A 1 < r < n) 

Holds{At{ri), s) A Holds{At{r 2 ), s) D ri = T2 

Holds{Empty{b), s) D ^Holds{Carries{b, r), s) Al <h <k , , 

Holds\Carries{b,r),s) 1 <b <k A 1 <r <n 
Holds{Carries{b,ri), s) A Holds{Carries{b,r 2 ), s) D ri=r 2 
Holds{Request{ri,r 2 ), s) D ri 7^ r-2 A 1 < ri, r2 < n 

Here, n is the number of offices and k the number of mail bags. Put in words, 
in every situation the robot is at some unique office (first and second constraint), 
mail bags cannot be both empty and carrying a package (third constraint), the 
contents of a mail bag must lie within the allowed range and be unique (fourth 
and fifth constraint), and packages need to have different sender and addressee 
in the allowed range (last constraint). It is easy to verify that, e.g., state (1.1) 
satisfies these domain constraints wrt. the unique-name axioms given earlier. □ 

Fluents are sometimes called functional if the domain constraints stipulate 
that they are unique in some of their arguments, as with At(r) in our example. 
This is because it is quite natural to say that “in situation s, the location of the 
robot is 1” instead of “in situation s it holds that the robot is at location 1.” 



1.3 State Update Axioms 

One purpose of the internal model is to help the agent decide which action it can 
perform at a certain stage. Our mailbot, for example, shall drop and pick up 
packages whenever possible. This requires the robot to know that it can deliver 
only packages which it carries and only if it is at the recipient’s office. Similar 
conditions apply to the other actions of the mailbot. So-called precondition 
axioms are used to formally specify the circumstances under which an action is 
possible in a state. 



Definition 1.12 Consider a fluent calculus signature with action functions A, 
and let A G A. A precondition axiom for A is a formula 

Poss{A{x), z) = n(z) 



where H(z) is a state formula with free variables among a;, z. 



□ 



Example 1 (continued) The preconditions of the three actions of the mailbot 
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can be formalized by these axioms: 

Poss{Pickup{b,r), z) = 

Holds {Empty {b), z) A 

(3ri) {Holds{At{ri), z) A Holds{Request{ri, r) , z)) 

Poss{Deliver{b), z) = (1-10) 

(3r) {Holds{At{r), z) A Holds{Carries{b, r),z)) 

Poss{Go{d), z) = 

(3r) {Holds{At{r), z) A [d= Up A r < n V d = Down A r > 1 ]) 

With this specification, it is a logical consequence that, say, in the initial 
state (1.1) these actions are possible: Pickup{b,r) where b £ {1,2,3} and 
r G {2, 3, 5}, and Go{Up). □ 

The assertion that an action and a sequence of actions, respectively, is possible 
in a situation reduces to the corresponding assertion about the states: 

Poss{a,s) “= Poss{a, State{s)) 

Poss([ai, . . . , On], s) Poss{ai, s) A ... A Poss{an, Do{[ai, . . . , a„_i], s)) 

In order to keep their internal model up to date, agents need to know not 
only the preconditions of their actions but also how these affect the environment. 
With this knowledge the agent can update its model after each action to reflect 
the changes it has effected. The mailbot, for example, needs to know that 
picking up packages in mail bags causes the packages to be carried. It also 
needs to know that mail bags cease to be empty when being filled, for otherwise 
the robot would be tempted to All it again. On the other hand, collecting a 
package does not alter the contents of the other bags, nor does it change the 
location of the robot, etc. 

The changes that an action causes are specified by axioms which define the 
update of states. Under the condition that the action A{x) in question is 
possible in a situation s, the so-called state update axiom for this action relates 
the resulting state, State{Do{A{x),s)), to the current state, State{s). More 
precisely, the positive and negative effects of the action are cast into an update 
equation which encodes the difference between these two states. Some actions 
have conditional effects, like Go{d) in the mail delivery world where the effect 
depends on whether d = Up or d = Down . State update axioms may therefore 
combine several update equations, each of which applies conditionally. 

Definition 1.13 Consider a fluent calculus signature with action functions A, 
and let A & A. A state update axiom for A is a formula 

Poss(A(x),s) D 

{3yi) {Ni{s) A State{Do{A{x).,s)) = State{s) — III + D'l) C 111 

V ... V 

(3y„) (A„(s) A State {D o {A{x) , s)) = State{s) — -I- 



where 
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• n > 1 ; 

• for z = 1, . . . , n, 

— Ai(s) is a situation formula with free variables among x, s; 

— are finite states with variables among x,yi. 

The terms and z?” are called, respectively, positive and negative effects 
of A{x) under condition Ai(s). An empty positive or negative effect is usually 
not explicitly stated. □ 

Example 1 (continued) The effects of the three actions of the mailbot can 
be formalized by these state update axioms: 

Poss{Pickup{b,r), s) D 
(3ri) (Holds{At{ri), s) A 

State{Do{Pickup{b,r), s)) = 

State{s) — Empty{b) o Request{ri,r) + Carries{b,r ) ) 

Poss{Deliver{b), s) D 
(3r) {Holds{At{r), s) A 

State{Do{Deliver{b), s)) = ... 

State{s) — Carries {b, r) + Empty [b ) ) \ ■ J 

Poss{Go{d), s) D 

(3r) (d = Up A Holds{At{r), s) A 

State {Do{Go{d), s)) = State{s) — At{r) + At(r + 1) ) 

V 

(3r) {d = Down A Holds{At{r), s) A 

State {Do{Go{d), s)) = State{s) — At{r) + At{r — 1) ) 

Take, for instance, S'tate(S'o) = (1.1), then Poss{Pickup{l,2), Sq) according to 
the precondition axiom in (1.10). Let Si = Do{Pickup{l, 2), 5'o), then the state 
update axiom for picking up packages implies 

State(Si) = S tate {S o) — Empty (1) o Request (1,2) + Carries (1,2) 

A solution to this equation (more precisely, to the set of equations obtained by 
macro expansion) is given by 

State(Si) = At{l) o Empty{2) o Empty{3) o Carries{l, 2) o 

Request{l, 3) o . . . o Requested, 4) 

The reader may verify that likewise the precondition and state update axioms 
entail Poss{Go{Up), Si) and, setting S 2 = Do{Go{Up),Si), 

State{S 2 ) = At{2) o Empty{2) o Empty{3) o Carries{l, 2) o 

Request{l, 3) o . . . o Request{6, 4) 
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Finally, the axiomatization implies that Poss[Deliver(l), S 2 ) and, setting 83 = 
Do{Deliver{l), S 2 ), 

State{S^) = At{2) o Empty{l) o Empty{2) o Empty{3) o 

Request{l, 3) o . . . o Request{&, 4) 

We could go on with this derivation to prove that Figure 1.4 (page 13) indeed 
depicts the correct state for the given situation. □ 

The concept of state update axioms tackles a classical problem in Artificial 
Intelligence, known as the frame problem. It arises because mere effect for- 
mulas do not suffice to draw conclusions about unchanged fluents. Suppose, for 
example, action Pickup were specified solely by the effect axiom 

Poss{Pickup{b,r), s) D 

(3ri) {Holds{At{ri),s) A ^Holds{Empty{b), Do{Pickup{b,r), s)) 

A -^Holds{Request{ri,r) , Do{Pickup{b, r), s)) 

A Holds{Carries{b, r), Do{Pickup{b, r), s )) ) 

Substituting {5/1, r/2, s/S'o}, this formula alone does not entail whether At{l) 
or Empty {2) or any other initially true fluent still holds in the resulting situ- 
ation. This is in contrast to the state equations used in state update axioms, 
which do imply that the “frame,” i.e., the unaffected fiuents, continue to hold 
after performing the action. In other words, a fluent holds in the resulting sit- 
uation just in case it is a positive effect or it holds in the current situation and 
is not a negative effect. State update axioms thus constitute a solution to the 
frame problem. 

Theorem 1.14 Let and d~ he two finite states, then foundational axioms 
^sstate and State{Do{a, s)) = State{s) — d~ + entail 

Holds{f,Do{a,s)) = Holds{f,d^) 

V 

Holds{f,s) A ~^Holds{f,d~) 

Proof: Macro expansion implies the existence of a state z such that 

State{s) — d~ = z 
State{Do{a, s)) = z o d~^ 

According to Proposition 1.6, the first equation implies that Holds{f, z) = 
Holds{f,s) A ~^Holds{f,d~) . From the second equation and decomposition it 
follows that Holds{f, Do{a,s)) = Holds{f,z) V Holds{f,d~^). m 

State update axioms complete the formal specification of agent domains. 
The various components are summarized in the following definition. 

Definition 1.15 Consider a fluent calculus signature with action functions A. 
A domain axiomatization in special fluent calculus is a finite set of axioms 
E — U EpQss U Esua C E,iux C E^nu U Esstate where 
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• Erfc set of domain constraints; 

• Sposs set of precondition axioms, one for each A ^ A] 

• Esua set of state update axioms, one for each A^ A] 

• Eaux set of auxiliary axioms, with no occurrence of states except for 
fluents, no occurrence of situations, and no occurrence of Poss; 

• Einit = {State (So) = ry} where To is a ground state; 

• Esstate foundational axioms of special fluent calculus. □ 

It is useful to have some guidelines of how to arrive at “good” domain ax- 
iomatizations. In particular, state update axioms should be designed in such 
a way that they never leave doubts about the effects of a (possible) action. 
To this end — at least for the time being — domain axiomatizations should be 
deterministic in the following sense. ^ 

Definition 1.16 A domain axiomatization E is deterministic iff it is con- 
sistent and for any ground state r and ground action a there exists a ground 
state r' such that 

E \= [State{s) = T A Poss{a, s)] D State{Do{a, s)) = A 



□ 

By this definition, the state update axioms in a deterministic domain entail 
unique successor states for any state and any action that is possible in that 
state. 

The following theorem tells us how to design deterministic domains. Basi- 
cally, it says that the conditions in state update axioms should be set up in 
such a way that there is always a unique instance of one and only one update 
equation that applies. Furthermore, this equation should always determine a 
successor state that satisfies the domain constraints. 

Theorem 1.17 Consider a fluent ealculus signature with action functions A, 
then a domain axiomatization E is deterministic if the following holds: 

T U E(Iq U Edy^.j; U E^jiii IS consistent^ 

2. Eaux 1= Ai{x) A ^ 2 (j 7 ) for all Ai, A 2 G A such that Ai A ^ 2 ; 

3. for any ground state r and ground action A(r) with state update axiom 

( 1 . 11 ) if 

(a) T satisfies E^c wrt. Eaux ond 

(b) U |^ .^^-^^ (^(^) 5 "^) ; 

^ Nondeterminism is going to be the topic of Chapter 7. 
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then there is some i = I, . . . ,n and ground terms t such that 

(c) Faux h Aj(r){f/f,27j/r} A [Ai(T){f/f} A yi = t\, 

(d) Faux h Aj{T){x/r} for all j i, and 

(e) any state t' with Esstate U Eaux \= t' = t - df + df {x/r,yi/t} 
satisfies E^^ wrt. Eaux- 

Proof: Let s be a situation, r a ground state, and A{r) a ground action. 

Suppose State{s) = r and Poss{A{f)^s), then we have to show that there 
exists a ground state t' such that S entails State{Do{A{f),s)) = t': 

From Poss{A{r), s) and State{s) = r it follows that Poss{A{r),T) . Ac- 
cording to conditions (c) and (d), there is a unique i = 1, . . . ,n and a unique 
instance t for ifi such that Eaux entails Ai(r){x/f', }. Hence, the state 
update axiom for A(x) implies 

State{Do{A{x) , s)) = t — d~ + df {x/f, iji/t} 

This shows that E entails State{Do{A{r), s)) = t' for some ground state t' . 

To prove that S is consistent, consider a model l for the set of axioms 
E' = Egatate U Eaux- Since neither of these axioms mentions predicate Pass, 
since a precondition axiom alone cannot be inconsistent, and since any two 
actions starting with different function symbols cannot be equal, there is also 
a model t for E' U E^oss- Moreover, since neither of these axioms mentions 
situations, a model for E = E' U Epogg U Emit U Egua U E^c can be constructed 
from i as follows. Let State{So) be interpreted by a term ry as stipulated 
by Einit- For any ground situation a such that State{a) is interpreted in t by 
a ground state r which satisfies E , and for any ground action A(r) such that 
Poss{A{r),T) is true in t, let State{Do{A{r),a)) be interpreted by a state r' 
which satisfies condition (e). So doing determines a model for Egua according 
to conditions (c) and (d), and for Edc according to condition (e) and since 
Einit U Edc is consistent under Eggtate U Eaux according to assumption (a). ■ 

The value of this theorem lies in reducing overall consistency to consistency of 
the auxiliary axioms and domain constraints, and to a local condition on state 
update axioms. Both are usually easy to verify. 

Example 1 (continued) Let the mailbot domain consist of domain con- 
straints (1.9); precondition axioms (1.10); state update axioms (1.12); auxil- 
iary axioms UNA[At, Empty, Carries, Request] , UNA[Pickup, Deliver, Go] , and 
UNA[Up, Down]; and initial state axiom State{So) = (1.1). This domain is de- 
terministic: 

1. Since state (1.1) satisfies the domain constraints Edc wrt. the unique- 
name axioms Eaux-, the initial state axiom along with the foundational 
axioms, Edc, and Eaux are consistent. 

2. The unique-name axioms entail that different actions are unequal. 
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3. To verify item 3 of Theorem 1.17, let r be a state which satisfies the 
domain constraints of the mail delivery world, and consider the three 
actions of the mailbot: 

• Suppose Poss{Pickup{b,r),T). The domain constraints imply that 
there is a unique ri such that Holds{At{ri),T). Let 

t' = T — Empty{b) o Request{ri,r) + Carries{b,r) 

Since t satisfies the first two domain constraints in (1.9), so does 
t' . The third, fourth, and fifth domain constraints are satisfied in 
r' wrt. b since Carries{b,r) holds and Empty{b) does not. These 
three constraints are satisfied in t' wrt. any other bag since they are 
satisfied in r. Finally, since r satisfies the last domain constraint, 
so does t' . 

• The argument for Deliver(b) is similar. 

• Suppose Poss{Go{d).,T). The domain constraints imply that there is 
a unique r such that Holds (At{r),T). Hence, as Up ^ Down, only 
one condition applies in each instance of the state update axiom. Let 

t' = T — At{r) + At{r ± 1) 

The first two domain constraints are satisfied in t' since they are 
satisfied in r and since the precondition for Go{d) ensures that 
l<r±l<n. As T satisfies the other domain constraints, so does 
t'. □ 

Deterministic domains obey a property which is essential for agent program- 
ming based on special fluent calculus: If an agent starts off with complete state 
information and performs possible actions only, then the axiomatization entails 
complete knowledge in every situation reached along the way. 

Corollary 1.18 Let E be a deterministic domain axiomatization. Gonsider 
a ground situation u = Do{[a\, . . . ,an], Sq) with E ^ Poss([ai, . . . , a„], Sq), 
then there exists a ground state r such that E |= State{a) = r. 

Proof: By induction on n. If n = 0, then a = Sq and the claim fol- 
lows from initial state axiom = {State(So) = tq} in E. Suppose the 

claim holds for n, and consider a situation Do(q!„+i, Do([ai, . . . , a„], 5'o)). 
The induction hypothesis implies the existence of a ground state r such that 
E 1= State{Do{[ai, . . . , an], So)) = r; hence. Definition 1.16 and assumption 
Poss(a„+i, Do([q!i, . . . , a„], S'o)) imply the claim. ■ 

Domain axiomatizations using fluent calculus set the stage for agent pro- 
grams. Precondition and state update axioms provide the background knowl- 
edge which allows agents to maintain an internal model of their environment. 
The foundational axioms of fluent calculus define how conditional statements 
in agent programs are evaluated against the internal model. In the upcoming 
chapter, we cast both the foundational axioms and the domain-dependent ones 
into a logic program, and we show how to write agent programs on top of it. 
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1.4 Bibliographical Notes 

The roots of formal logic trace back to an analysis of the structure of logical 
arguments by the Greek philosopher Aristotle in his treatise Organon (that 
is, “instrument”). The foundations for modern predicate logic have been laid 
in [Frege, 1879]. There are numerous textbooks providing gentle introductions 
to first-order logic, among which is the classical [Quine, 1982]. Most of the logic 
notation used in this book has been taken from [Davis, 1993], where it has also 
been shown that the use of sorts and the special treatment of equality can in 
fact be reduced to standard predicate logic. 

The idea to build intelligent agents using logic has been pursued ever since 
the emergence of Artificial Intelligence, beginning with [McCarthy, 1958]. The 
oldest formalism for axiomatizing actions and effects is situation calculus, 
which has introduced the basic notions of fluents, actions, and situations [Mc- 
Carthy, 1963]. The fundamental frame problem has been uncovered in [Mc- 
Carthy and Hayes, 1969] in the context of situation calculus. In [Green, 1969], 
situation calculus is applied to problem solving using a system for automated 
theorem proving. Lacking a good solution to the frame problem, this system is 
limited to domains with a small state space and short action sequences. 

Trading the expressivity of situation calculus for efficiency, STRIPS (for: 
Stanford Research Institute Problem Solver) [Fikes and Nilsson, 1971] is re- 
stricted to complete states in using set operations to calculate the result of 
actions. With the aim of combining the expressivity of logic with the efficiency 
of STRIPS, several non-classical logics have been proposed as solutions to the 
frame problem [Bibel, 1986; Masseron et al., 1993; Bibel, 1998]. Combining the 
fundamental notion of states with that of situations traces back to the logic 
programming-based approach of [Holldobler and Schneeberger, 1990]. Char- 
acterizing states as multisets of fluents (where, as opposed to sets, multiple 
occurrences of elements count), this formalism has been shown equivalent to 
the aforementioned non-classical approaches for a particular kind of planning 
problems [GroBe et al., 1996]. The paper [Holldobler and Kuske, 2000] contains 
some decidability results for this approach. 

The foundations for fluent calculus have been laid in [Thielscher, 1999] with 
the introduction of state update axioms as a combination of a state-based rep- 
resentation with elements of situation calculus. The signature of fluent calcu- 
lus is an amalgamation of the axiomatization of states as in [Holldobler and 
Schneeberger, 1990] and the variant of situation calculus presented in [Pirri and 
Reiter, 1999]. The full axiomatic theory of fluent calculus has first been pre- 
sented in [Thielscher, 2001c], based on investigations carried out in [Storr and 
Thielscher, 2000]. As long as fluent calculus is applied to complete states only, 
it is essentially a reformulation of STRIPS in classical logic. 



1.5. EXERCISES 



23 



1.5 Exercises 

1.1. Prove that foundational axioms E^state entail 

(a) Z O 0 = z; 

(b) zo z = z. 

1.2. Prove that foundational axioms Esstate are mutually independent. 

Hint: Find a model for the negation of each axiom with all other axioms 
satisfied. 

1.3. Let T = /i o . . . o /„ be a finite state. Prove that foundational axioms 
Esstate entail 

(a) S!^^.^Holds{fi,z) D zoT = z\ 

(b) A7=i^Holds{fi,z) D z-T = z. 

1.4. Suppose our mailbot has just one bag, and consider the initial situation 
where the robot is at room 2 with an empty bag and two requests, from 1 
to 5 and from 3 to 4, respectively. Find the situation with the fewest 
Go actions in which the two packages have been delivered, and prove that 
in this situation no requests are left and all bags are empty. 

1.5. Suppose that initially there are only two requests in the mail delivery world 
with a robot that has three mail bags, and all mail bags are initially empty. 
Show that there is no ground situation a = Do{[ai, . . . , a„], Sq) such that 
the mailbot domain axiomatization entails (V6) -^Holds{Empty{h)^a). 

1.6. Suppose the mailbot can temporarily park packages at offices different 
from their destination. 

(a) Replace the binary fluent Request{r i,V 2 ) by the ternary variant 
Request{ri,r2,rA with the meaning that in room ri there lies a 
package addressed from room T2 to ra. Rewrite the domain con- 
straints and the precondition and state update axioms accordingly. 

(b) Design precondition and state update axioms for a new action called 
Putdown{b) of putting down the contents of mail bag h at the current 
location. Show that the modified domain axiomatization, too, is 
deterministic. 

(c) Solve Exercise 1.4 in the modified domain. 

1.7. Design the environment of a software agent which transfers files to remote 
locations. 

(a) Define domain constraints for the fluents Request{addr, file ^ size), 
indicating that the file named fie of size size should be moved to 
the remote location addr; and Tar File {addr, size), indicating that 
an archive file of total size size exists, to be sent to the remote site 
addr . 
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(b) Define precondition axioms and state update axioms for the actions 
Tar (addr, file), for adding file to the archive file for remote site 
addr] and Ftp(addr), for transferring the archive file for addr. The 
latter action shall have the side effect of removing the file that has 
been sent. Assume that the size of the archive file is simply the sum 
of the sizes of the files it contains. Furthermore, assume that an 
archive file can be safely ftp’ed only if it has less than one megabyte 
(1, 048, 576 bytes). 

(c) Consider the initial state 

State(So) = Request{ ‘ftp:/ /photo. de” , 510000) o 

Request{ “ftp://photo.de” , “img2.jpg” ,180000) o 
Request^ “ftp://photo.de” , “img3.jpg” ,110000) o 
Request\ “ftp://photo.de” , “imgf.jpg” ,810000) o 
Request^ “ftp://photo.de” , “img5.jpg”, 150000) o 
Request\ “ftp://photo.de” , “img6.jpg”, 120000) 

Find a situation a such that the domain axiomatization entails 
State (a) = 0, and prove this. 

(d) Show that if an initial request concerns a file larger than 1 MB, 
then there is no ground situation a = Do([ai, . . . , a„], Sq) such that 
State{a) = 0 is entailed. 



Chapter 2 



Special FLUX 



FLUX, the FLUent eXecutor, is a programming language based on the ideas 
of fluent calculus. It allows to write concise and modular programs for agents 
which base their decisions on an explicit world model. In order to keep this 
model up to date during the execution of the program, every agent program 
requires an encoding of the underlying domain axiomatization, in particular the 
state update axioms. Due to the declarative nature of FLUX, domain axioms 
in fluent calculus translate directly into executable program statements. 

FLUX agents are implemented on top of a kernel, which endows robotic 
agents with general reasoning capabilities for maintaining and using their in- 
ternal model. As a logic program, the kernel can be easily verified against 
the semantics of fluent calculus. In a similar fashion, formal methods can be 
applied to the agent programs themselves in order to prove their correctness, 
which is of particular interest if agents handle secure information or are used 
for safety-critical control tasks. 

In this chapter, we introduce the special FLUX language and system, which 
is suitable for programming agents with complete state information. As we will 
see, high-level agent control programs can be divided into three layers, which 
we introduce in a bottom-up manner; the last section deals with the interaction 
with other agents and in particular with users: 

2.1. A domain-independent kernel program endows the agent with basic 
reasoning facilities. 

2.2. On top of the kernel, the domain encoding provides the agent with the 
necessary knowledge of its actions and environment. 

2.3. Finally, the control part defines the intended, goal-oriented behavior of 
the agent. 

2.4. If the state of its environment is not under the complete control of an 
agent, then its program needs to be extended to allow for exogenous 
actions, such as the addition and cancellation of requests to the mailbot. 
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Perform(a) 




Physical Agent 



Figure 2.1: Architecture of agent programs written in special FLUX. 



2.1 The Kernel 

The general architecture of a FLUX agent is depicted in Figure 2.1. The pro- 
gram Pstrategy implements the behavior of the agent with the help of two basic 
commands defined in the kernel program: The expression Holds{f, z) is used 
to condition the behavior of the agent on the current state of the environment. 
The predicate means exactly what it does in fluent calculus: It is true if and 
only if condition (i.e., fluent) / is true in the current world model (i.e., state) 2 . 
Secondly, the control program Pstrategy uses the statement Execute{a, z\, Z 2 ) 
to trigger the agent to actually perform an action a in the outside world and, 
simultaneously, to update the current state zi to Z 2 according to the changes 
effected by action a. The actual performance of an action needs to be pro- 
grammed via Performia). 

The update computation itself relies on the domain specification Pdomain- 
In particular, the effects of actions are defined by state update axioms. To facil- 
itate these specifications, the FLUX kernel provides a definition of the auxiliary 
predicate Update {zi, ,D~ , Z 2 )- Its intuitive meaning is that state Z 2 is the 
result of positive and negative effects and d~ , respectively, in state zi. In 
other words, the predicate encodes the update equation Z 2 = zi — . 

Syntax and semantics of both agent programs in FLUX and the kernel derive 
from the paradigm of logic programming. We appeal to the standard completion 
semantics and negation-as-failure computation rule. Agent programs frequently 
utilize the “cut,” whose operational semantics is given by a reduction of the 
computation tree. The programs in this book are all implemented using the 
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logic programming language Eclipse.^ 



A logic program is a finite sequence of clauses of the form p{t ) <— Li, . . . , 
where p{t ) is an atom and Li, . . . ,Ln are literals, that is, atoms or negated 
atoms (n > 0). The semantics of a program is to regard all clauses with 
the same leading predicate p as the definition of p. Formally, let p be any 
predicate symbol of the underlying signature such that 

p{ti) ^ Lii, . . . , Lirn 

p{lm') ^ ^ml: ■ ■ ■ ; ^mrim 

are the clauses for p in a program P (m > 0). Take a sequence x of pairwise 
different variables not occurring in any of the clauses, then the definition for 
p in P is given by the formula 

m 

p(x) = \J (x — A L^\ A ... A ) 

i=l 

where the y,i^s are the variables of the respective clause. The completion of a 
program P, written COMP [P], is the definitions of all predicates in P along 
with unique-name axioms for all function symbols. 



Special FLUX is suitable for the programming of agents which have com- 
plete information about the state of their environment, so that all computations 
concern ground states as in special fiuent calculus. A ground state is encoded as 
a list of ground fluents such that no fiuent occurs more than once. While states 
are (semantically) equal whenever they contain the same fluents, two lists are 
different if they contain the same fluents but in a different order. Hence, there 
is a one-to-many correspondence between states in fiuent calculus and lists of 
fluents in FLUX. 

Definition 2.1 Let (P, o,0) be a fiuent calculus state signature. Aground 
FLUX state is a list 

[/i, •■•,/«] (2.1) 

of pairwise different ground ffuents (n > 0). If r is a ground state, then by |r] 
we denote any ground FLUX state containing the same ffuents as r. □ 

In the following we will distinguish between a ffuent calculus state and its FLUX 
representation only when necessary. The initial state in our mailbot scenario 

^ The Eclipse© system was originally developed by the former European Computer- 
Industry Research Center (ECRC) in Munich, Germany, and is now maintained and 
distributed by the Center for Planning and Resource Control at Imperial College (Ic- 
Parc) in London, U.K. 
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holds (F, [F|_]). 

holds (F, Z) Z=[F1|Z1], F\==F1, holds (F, Zl) . 
holds (F. [F|Z], Z). 

holds (F. Z, [FlIZp]) Z=[F1|Z1], F\==F1, holds (F. Zl, Zp) . 

minus (Z, [] , Z) . 
minusCZ, [F|Fs], Zp) 

(\+ holdsCF, Z) -> Z1=Z ; holds(F, Z, Zl)), 
minus (Zl, Fs, Zp) . 

plus (Z , [] , Z) . 
plus(Z, [F|Fs], Zp) 

(\+ holdsCF, Z) -> Z1=[F|Z] ; holds (F, Z) , Z1=Z) , 
plus (Zl , Fs , Zp) . 

update (Zl, ThetaP, ThetaN, Z2) 

minusCZl, ThetaN, Z) , plus(Z, ThetaP, Z2) . 

execute(A, Zl, Z2) perform(A) , state_update (Zl , A, Z2) . 



Figure 2.2: Pskemei' The kernel of special FLUX. 



depicted in Figure 1.1, for example, may be encoded by the following clause, 
which defines an initial ground FLUX state: 

init(ZO) :- 

ZO = [ at(l) ,empty(l) ,empty(2) ,empty(3) , (2-2) 

request (1,2) , request (1,3) , . . . .request (6,4)] . 

The completion semantics of logic programs entails that any two syntac- 
tically different ground terms are unequal. Hence, FLUX makes the implicit 
assumption of uniqueness of names for all fluent functions of a signature, so 
that the list just given is indeed free of multiple occurrences, as stipulated by 
Definition 2.1. 

Based on the encoding of states as lists, the complete kernel of special FLUX 
is depicted in Figure 2.2. It consists of three parts: a definition for fluents to 
hold in states, a bunch of definitions culminating in a clause for updating a state 
by a list of positive and a list of negative effects, and a definition for executing 
an elementary action. Let this program be denoted by Pskemei ■ Its declarative 
semantics is given by the completion COMP[Pskemei\- In the following, we will 
discuss in detail each part of the kernel program. 

2.1.1 Holds/2 

The topmost two clauses in Figure 2.2 encode the assertion Holds{f,z), using 
the standard decomposition [Head \ Tail] of a list into its first element and the 
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rest list. The first clause says that if / occurs as the head of FLUX state z, 
then it holds in that state, and the second clause says that it holds in the state 
if it occurs in the tail of list z. The inequality condition f ^ fi in the second 
clause has been added for the sake of efficiency: Given that FLUX states contain 
no multiple occurrences, a fluent which is identical to the head cannot reappear 
in the tail. 



Logic programs are executed by queries. A query A, Q starting with an 
atom can be resolved with a program clause A' ^ Q' iff there exists a most 
general variable substitution 6 such that AQ = A' 0. As the resolvent one 
obtains the query {Q',Q)9. 

A computation tree for a program and query P U {<— Qo} is obtained as 
the limit of a sequence of trees, starting with query Qq. 

1. If the current query Q starts with an atom A, then Q has a finite 
number of resolvents as ordered children, one for each program clause 
whose head unifies with the atom A, and the new current query is the 
leftmost child. Q fails if no clause is applicable in this way, and the 
new current query is the next unsolved query in the tree. 

2. If Q is of the form then the negated first sub-goal is treated by 

the principle of negation-as-failure: Q is linked to a subsidiary tree 
consisting of just the root Qi , which becomes the new current query. 
If later on all branches of the subsidiary tree are failure nodes, then Q 
itself has Q' as the only child. Otherwise, if the subsidiary tree contains 
a finite branch ending with the empty query, then Q itself fails. 

The process of extending a tree may go on forever, in which case the limit is an 
infinite computation tree. A successful derivation in the main tree ends with 
the empty query. The answer computed by a successful derivation consists of 
a substitution for the variables in the query which is obtained by combining 
all substitutions used for resolving the queries along the branch. 



As an example, recall the clause for the initial state in the mail delivery 
scenario given earlier, (2.2), and consider the following query, which determines 
all initial requests issued from the office where the robot is currently located: 

?- init(Z), holds (at (R) , Z) , holds(request(R,Rl) ,Z) . 

R = 1 
R1 = 2 

Z = [at(D ,empty(l) , . . .] More? 

R = 1 
R1 = 3 
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Z = [at (1) , empty (1) ,... ] More? 

R = 1 
R1 = 5 

Z = [at (1) , empty (1) ,... ] More? 
no (more) solution. 

Notice that the inequality in the second clause for Holds of the kernel does 
not apply when resolving the query Holds{Request{l,ri),[At{l), . . .]), since 
the term Request by virtue of containing a variable, is not identical 
to any element of the ground FLUX state. The computed answers are ob- 
viously correct under the semantics of fluent calculus, as Esstate along with 
UNA\At^ . . . ^Request] entails 

Holds{At{r), Z) A Holds{Request{r, ri),Z) = 

r=lAri=2 V r = lAri = 3 V r=lAri=5 

if Z is the term (1.1) at the beginning of Chapter 1 (page 5). 

Using negation (which is written “\+” in Prolog) the next sample query 
computes the nearest destination of a package waiting at the current location 
of the robot: 

?- init(Z), holds (at (R) ,Z) , holds (request (R, Rl) ,Z) , 

\+ ( holds(request(R,R2) ,Z) , abs(R-R2) < abs(R-Rl) ). 

R = 1 
Rl = 2 

Z = [at (1) , empty (1) ,... ] More? 
no (more) solution. 

Put in words, ri is the nearest requested destination if there is no requested 
destination t -2 strictly closer to the current location, r. Again, it is easy to see 
that the resulting answer is supported by fluent calculus. 

The following theorem shows that the correspondence is not coincidental 
between the computed answers in the two examples and the underlying fluent 
calculus semantics. The clauses of the kernel which define Holds{f, z) are 
provably correct in general. 

Theorem 2.2 Consider a fluent caleulus state signature with fluent functions 
T , and let ip he a fluent and t a ground state, then for any |r], 

COMP[Pskernel] \= Holds{ip, {t]) ijf Esstate C UNA[P] ^ Holds{ip,T) 



and 



COMP[Pskernel] h ^ Holds{ip, {t}) iff E sstate C UNA[E] ^ ~^Holds{ip,T) 
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Proof: The completion COMP[Fgkf^mei\ entails the definition 

Holds{f,z) = (3z') z = [/ I z'] 

V (2.3) 

(3/i, zi) (z = [/i I zi] A / /i A Holdsif, zi)) 

Let |r] = (n > 0), then Esstate h r = /i o /2 o . . . o /„ 

according to the foundational axioms of associativity and commutativity, and 
-^Holds{fi, f '2 o .. . o /„) according to Definition 2.1 and Proposition 1.5. De- 
composition and irreducibility imply that Egstate U UNA[E] entails 

Holds{(fi, t) = (fi = f I y if ^ fi A Holds{ip, /2 o ■ ■ ■ o /™) 

which, along with the instance {//<^, z/[/i, / 2 , . . . , /«]} of (2.3), proves the 
claim. ■ 

2.1.2 Update/4 

Predicate C/pdate(zi, ■d+, 1 ?“, Z 2 ) encodes the equation Z 2 = z\ — . Its 
definition is based on the predicates Minus{z,d~ , z') and Plus{z,d^ , z') , which 
encode, respectively, the macros z' = z — and z' = z . The former in 
turn uses the ternary Holds{f, z, z'), whose intended semantics is Holds{f, z) A 
z' = z — f. The definition of Minus follows closely the corresponding macro 
expansion as given in Chapter 1. The encoding of Plus uses a case distinction, 
too, making sure that no multiple occurrences are introduced when adding a 
fluent to a list. 



Clauses of logic programs, and hence resolvents, may include the cut, writ- 
ten “!”. The unique resolvent of a query Q of the form \,Q' is Q' , but the 
side effect of this resolution step is that in the computation tree all branches 
to the right of the one leading to Q are cut off. We will frequently use the 
standard syntax Qi ^ Q 2 in clause bodies, which stands for Qi, !, (52- 
Syntactic sugar allows for nested negation by defining <— Qi,^{Q),Q 2 as 
^ Qi,~’ 0 ,Q 2 along with the clause o <— (5 where “o” is a predicate symbol 
not occurring elsewhere. We will also use disjunction in clauses, written 
p{t) ^ Q,{Qi',Q 2 ),Q' , which is defined as p{t) ^ Q,o along with the two 
clauses o ■<— Qi,Q' and o ^ Q 21 Q' ■ 



As an example, consider the update of the initial state in our mail delivery 
scenario by the positive effects At (2) o Empty {1) (the latter of which happens 
to be true) and the negative effects At{l) o Request{6,5) (the latter of which 
happens to be false): 

?- init(ZO), updateCZO, [at(2) ,empty(l)] , 

[at(D ,request(6,5)] ,Z1) . 
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ZO = [at(D ,empty(l) ,empty(2) ,empty(3) , 

request (1,2) , request (1,3) , . . . , request (6, 4)] 

Z1 = [at(2) ,empty(l) ,empty(2) ,empty(3) , 

request (1,2) , request (1,3) , . . . , request (6, 4)] 

It is easy to verify that the second FLUX state is indeed a list |ti] such that 
Ti = To — At{l) o Request{2, 1) + At{2) o Empty{l), where [tq] is the initial 
FLUX state of clause (2.2). 

Correctness of the FLUX definition of update is established by a series of re- 
sults concerning the auxiliary predicates, showing that each is defined in Pskemei 
according to the intended meaning. To begin with, the following lemma shows 
that the ternary Holds{f, z, z') is true just in case / holds in z and z' is z 
without /. 

Lemma 2.3 Consider a fluent calculus state signature with fluent functions 
T , and let ip he a fluent and r, t' he ground states, then for any |r] there 
exists |r'] such that 



COMP[Pskernel] \= Holds{p, [t] , IPj) 

tff 

^sstate u UNA[E] fy Holds {if, t) /\t' = T - if 
Proof: The completion COMP\Pskemei] entails the definition 

Holds{f,z,z') = z=[f\z'] 

(3/i,zi,z")(^'=[/i|^"] Az=[/i|zi]A 
/fy/i A Holds{f,zi,f')) 

Let |r] = [/i, / 2 , ■■■Jn] {n> 0), then UNA[T] fy fy f fy for any i fy y , and 
Esstate H T = f I o f'2 o ... o f^ accoi'ding to the foundational axioms of asso- 
ciativity and commutativity. Decomposition, irreducibility, and Proposition 1.6 
imply that Egstate U UNA[iF] entails 

Holds{(p, t) At' = T - ip = </? = /i A r' = fy o . . . o /„ 

V 

i3z"){T' = hoz" ApfhA 
Holds{p,f2 o . . . o /„) A 
z" = fio ...o fn-p) 

which, along with the instance {f jp, -z/[/i, /fy ■ • ■ , /n]j of (2-4), proves 

the claim. ■ 

The conditions of this lemma should be carefully observed: Among all FLUX 
states |r'] which encode r' only one satisfies |r], |r']) . This is due 

to the fact that the fluents in the list |r'] occur in the very same order as in 
the list |r]. Thus, for example. 
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?- holds(f2, [fl,f2,f3] , [fl,f3]) . 
yes. 

?- holds(f2, [fl,f2,f3] , [f3,fl]) . 
no (more) solution. 

This restriction propagates to the definition of predicate Minus , which is verified 
next. 

Lemma 2.4 Consider a fluent caleulus state signature with fluent functions 
T, and let r,'d~ ,t' he ground states, then for any |r] and |i?“] there exists 
|r'] such that 



COMP[Pskernel] h MnWS (|t] , |l) Mt'D 

€ 

^sstate U UNA[T] = T-d- 

Proof: The completion C'OMP[T’sfcernez] entails the definition 

Minus {z, z~ , z') = 

= [] A z' = z 

(3/,zf,zi)(z" = [/|zf] A 

( -^Holds{f, z) A Zi = z V Holds{f, z, Zi) ) A 
Minus {zi, zf , z') ) 

Let |d"l = [/i, . . . , then Esstate h = /i ° /2 o ■ ■ • o /„ according to 
the foundational axioms of associativity and commutativity. The proof is by 
induction on n. 

If n = 0, then d =0 and ] = []• The instance {z/|r],z‘ /[]} of (2.5) 
implies 

Minus{{T\, [],z') = z' = |r] 

which proves the claim since z — % = z' is equivalent to z = z' by definition. 
In case n > 0, the instance z“/|z?“]} of (2.5) implies 



Minus{lTj,[fi,f 2 ,---,fn],z') = 

(3zi) ((^ZfoZds(/i,|r]) Azi = |t] V fl‘oZds(/i, |t], Zi) ) A 
Minus{zi,[f 2 , . . . , fri], z') ) 

which proves the claim according to Theorem 2.2, Lemma 2.3, the macro defi- 
nition for and the induction hypothesis. ■ 

Again, the order in which the fluents occur in the resulting FLUX state is 
determined by their order in the original list. For example. 
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?- minus([fl,f2,f3] , [f2,f4] , [fl,f3]) . 
yes. 

?- minus([fl,f2,f3] , [f2,f4] , [f3,fl]) . 
no (more) solution. 

In a similar fashion, the clauses for Plus are correct, as the following lemma 
shows. 

Lemma 2.5 Consider a fluent calculus state signature with fluent functions 
T, and let r, he ground states, then for any |r] and there exists 

|r'] such that 

COMP[Pskernet] h H , |^+1 , |t'1 ) 

iff 

EsstateU UNA[E] ^ t' =T0d+ 

Proof: Exercise 2.2 ■ 

Obviously, the order in which the fluents occur in the resulting state is deter- 
mined by their order in both the original list and the list of fluents to be added. 
For example, 

?- plus([fl,f2,f3] , [gl,g2] ,Z) . 

Z = [g2,gl,fl,f2,f3] 
no (more) solution. 

The intermediate results culminate in the following theorem, which shows 
that the encoding of update is correct. Naturally, the definition inherits the 
restriction which applies to the auxiliary predicates so that just one particular 
representative of the updated state is inferred. That is to say, the definition is 
correct for computing the result of a state update rather than for verifying that 
two states satisfy an update equation. This is in accordance with the intended 
use of Update{zi,'d'^ , Z 2 ) in agent programs, namely, to call it with the 
first three arguments instantiated and the last argument left variable. (See 
Exercise 2.3 on a more general definition of Update, which allows to infer all 
representatives of the resulting state.) 

Theorem 2.6 Consider a fluent calculus state signature with fluent functions 
T, and let ti, , d~ , and T 2 be ground states, then for any |ri], and 

I'd”], there exists |t 2 ] such that 

COMP[Psker~nel] h Ppdate ( [nl , |l)+l , [^"l , |t 2 ] ) 

iff 

Esstate U UNA[P] ^ T 2 =Ti-d-+d+ 
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Proof: The completion COMP[Pgkf^mei\ entails the definition 

Update {zi, z~ , z~^ , Z 2 ) = {^z) {Minus {z\^ z~ , z) f\ Plus {z^ z'^ ^ Z 2 )) 

The claim follows from Lemma 2.4 and 2.5 along with the macro definition for 
update equations. ■ 

2.1.3 Execute/3 

The predicate Execute{a, Zi, Z 2 ) is defined so as to trigger the actual per- 
formance of elementary action a and, simultaneously, to update the current 
state zi to state Z 2 . The kernel program presupposes the definition of a 
predicate Perform{a) to cause the physical agent to carry out action a in 
the environment. Furthermore, the programmer needs to define the predicate 
StateUpdate{zi, a, Z 2 ) in such a way that it encodes the state update axioms for 
each action the agent may execute. 

2.2 Specifying a Domain 

The domain specification is an essential component of every FLUX agent pro- 
gram. In order to decide which of its actions are possible at a certain stage, 
the agent must know about the preconditions of actions. In order to set up 
and maintain a model of the environment, the agent needs to be told the initial 
conditions and the effects of every action. A domain specification in FLUX is 
obtained by a direct translation of a domain axiomatization in fluent calculus 
into logic programming clauses. 

Precondition Axioms 

An agent program should trigger the actual performance of an action only if the 
action is possible. A precondition axiom Poss{A{x), z) = II(z) translates into 
the logic programming clause 

Poss{A{x), z) ^ n(z). 

where II is a re-formulation of II using the syntax of logic programs. Provided 
that there is no other program clause defining Pass with first argument A{x), 
the “only-if” direction of the precondition axiom is implicit in the completion 
semantics. The encoding of the precondition axioms (1.10) (page 16) for the 
mail delivery domain, for example, are included in Figure 2.3, which depicts the 
full background theory for mailbot agents. 



State Update Axioms 

For the execution of an action, the FLUX kernel assumes that the predicate 
StateUpdate{zi, A{x), Z 2 ) has been defined in such a way as to reflect the update 
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poss(pickup(B,R) >Z) holds (empty (B) , Z) , 

holds (at (Rl) , Z) , 
holds (request (R1 ,R) , Z) . 

poss (deliver (B) , Z) holds (at (R), Z) , 

holds (carries (B, R) , Z) . 

poss (go (D), Z) holds (at (R), Z) , 

( D=up, R<6 ; D=down, R>1 ) . 

state_update (Z1 , pickup (B,R)> Z2) 
holds (at (Rl) , Zl) , 

update(Zl, [carries(B,R)] , [empty (B) .request (R1,R)] , Z2) . 

state_update (Zl , deliver(B), Z2) 
holds (at (R) , Zl) , 

update(Zl, [empty (B) ] , [carries(B.R)] , Z2) . 

state_update (Zl , go(D), Z2) 
holds (at (R), Zl) , 

( D = up -> Rl is R + 1 

; Rl is R - 1 ) , 

update(Zl, [at(Rl)], [at(R)], Z2) . 

Figure 2.3: Background theory for the mailbot program. 



of state Zl to Z 2 according to the effects of action A(x). Appealing to the 
kernel predicates Holds and Update, the encoding of a state update axiom of 
the form (1.11) (page 16) is straightforward using the following scheme: 

StateUpdate{z\, A{x), Z 2 ) <— Ai(zi), Update{z\,'d'l^ , z^) 

.^. ; ( 2 . 6 ) 
A,„(zi), Update {zi,d+,d-,Z 2 ). 

where the A^ ’s are logic programming encodings of the conditions A^ for the 
respective effects For the sake of efficiency, action preconditions are 

not verified when inferring the state update, since actions are assumed to be 
performed only in case the current state entails that they are possible. The ac- 
tual verification of precondition is left to the control program for an agent. The 
second part of Figure 2.3 shows the encoding of the state update axioms (1.12) 
(page 17) for the mailbot domain. 

Initial State 

Agents initialize their internal world model by a list representing the initial 
state. A standard way of encoding a domain axiom Einit = {State(So) = r} is 
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by the clause 

Init(zo) ^ zq = It]. 

for some FLUX state |r] representing r. An example is clause (2.2) in Sec- 
tion 2.1, which encodes the initial initial situation of the mailbot as depicted in 
Figure 1.1. 

Let P-maii be the FLUX domain specification of Figure 2.3, and consider the 
program PmaiiC{{2.2)}\JPskernei- The following sample query infers the state in 
situation Do{Deliver{l), Do{Go{Up), Do{Pickup{l,2), Sq))) , making sure that 
each action in the sequence is indeed possible at the time of performance: 

?- init (ZO) , 

poss (pickupCl , 2) ,Z0) , state_update(ZO, pickup (1,2) ,Z1) , 
poss(go(up) ,Z1) , state_update(Zl,go(up) ,Z2) , 

poss(deliverd) ,Z2) , state_update(Z2,deliver(l) ,Z3) . 

ZO = [at(D ,empty(l) ,empty(2) ,empty(3) , 

request (1,2) , request (1,3) , . . . , request (6, 4)] 

Z1 = [carries(l,2) ,at(l) ,empty(2) ,empty(3) , 

request (1,3) ,request (1 , 5) , . . . , request (6, 4)] 

Z2 = [at (2) , carries (1,2) , empty (2) , empty (3) , 

request (1,3) ,request (1 , 5) , . . . , request (6, 4)] 

Z3 = [empty(l) ,at(2) ,empty(2) ,empty(3) , 

request (1,3) ,request (1 , 5) , . . . , request (6, 4)] 

The reader may compare the result with the corresponding derivation in fluent 
calculus (page 17). 

Domain Constraints 

A set of domain constraints Ydc can be encoded in FLUX by a clause defining 
the predicate Consistent{z) so as to be true if state z is consistent wrt. the 
constraints. Let Ydc = {(Vs) Ai(s), ..., (Vs)A„(s)}, then the FLUX translation 
is ^ 

Consistent(z) <— Ai(z), ..., A„(z). 

where the A^’s are logic programming encodings of the A^’s. Domain con- 
straints (1.9) (page 15) of the mail delivery domain, for example, can be verified 
by the following clause, assuming a hallway with n = 6 offices and a robot with 
k = 3 mail bags: 

consistent (Z) :- 

holds (at (R) ,Z) , 1=<R, R=<6, 

\+ ( holds (at (Rl) ,Z) , holds (at (R2) , Z) , R1\=R2 ), 

\+ ( holds(empty(B) ,Z) , holds (carries (B, _) ,Z) ), 

\+ ( holds(empty(B) ,Z) , (B<1 ; B>3) ), 

\+ ( holds (carries (B, R) ,Z) , (B<1 ; B>3 ; R<1 ; R>6) ), 

\+ ( holds(carries(B,Rl) ,Z) , 
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holds(carries(B,R2) ,Z) , R1=\=R2 ), 

\+ ( holds (request (Rl, R2) ,Z) , (RKl ; Rl>6 ; 

R2<1 ; R2>6 ; R1=R2) ) . 

Our sample initial state (2.2) can then be proved consistent: 

?- init(ZO), consistent (ZO) . 

ZO = [at(D ,empty(l) ,empty(2) ,empty(3) ,request(l,2) , . . .] 

For the sake of efficiency, domain constraints need not be included in an 
agent program if they are enforced by other means. In particular, the FLUX 
specification of a domain which satisfies the assumptions of Theorem 1.17 does 
not require the consistency check after each action. 

Auxiliary Axioms 

Using standard logic programming, the universal assumption of uniqueness-of- 
names is built into FLUX so that these axioms need not be encoded separately. 
Other auxiliary axioms need to be encoded only if they are required for the 
precondition or state update axioms. 

2.3 Control Programs 

Control programs P strategy guide robotic agents over time. A typical program 
starts with the initialization of the internal model via the predicate Init{z). 
Executability of actions and other properties of the current state are determined 
using the kernel predicate Holds (f, z), and the actual execution of an action is 
triggered via the kernel predicate Execute{a, z\, Z 2 )- 

Example 1 (continued) The control program depicted in Figure 2.4 imple- 
ments the simple strategy for our mailbot which we have proposed at the very 
beginning, cf. Figure 1.2 (page 3). After initializing the internal world model, 
the program enters a main loop by which possible actions are systematically 
selected and executed. The order in which the actions are selected implies that 
the mailbot delivers packages as long as possible, then collects new packages as 
long as possible, followed by going up or down the hallway towards some office 
where it can pick up mail or for which it has mail. If the robot can neither 
deliver nor collect a package, and if there is no office to be visited, then the pro- 
gram terminates since all requests have been carried out. The reader may notice 
how the operator, i.e., the Prolog cut, is employed to avoid backtracking 
once an action has been executed. This is necessary since actions which have 
actually been performed in the outside world cannot be “undone.” 

Our simple mailbot agent does not care as to which concrete instance of a 
Deliver(b) or Pickup{b,r) action gets selected. Choosing whether to move up 
or down the hallway is also arbitrary if both directions are possible. Running 
the agent program with the initial state of Figure 1.1 (page 2), encoded as 
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main init(Z), main_loop(Z) . 



main_loop(Z) 

poss (deliver (B) , Z) 

poss (pickup (B, R) , Z) 

continue_delivery (D, Z) 

true . 



-> execute (deliver (B) , Z, Zl) , 
main_loop(Zl) ; 

-> execute(pickup(B,R) ) Z, Zl) , 

main_loop(Zl) ; 

-> execute (go (D) , Z, Zl) , 
main_loop(Zl) ; 



continue_delivery (D , Z) 

( holds (empty (_) , Z) , holds (request (R1 ,_) , Z) 



holds(carries(_,Rl) , Z) ), 
holds (at (R), Z) , 

( R < R1 -> D = up 

; D = down ) . 



Figure 2.4: A simple mailbot agent in FLUX. 



clause (2.2), the robot starts with collecting at office 1 the package for room 2 
and ends after exactly 80 actions with delivering to room 3 the package sent 
from room 6. 

To give an impression of the general performance of the program, the fol- 
lowing tables show the lengths of the generated action sequence for different 
scenarios of increasing size. The results are for maximal mail delivery prob- 
lems, that is, where requests are issued from each office to each other one. Hence, 
a scenario with n offices consists of (n — 1) • (n — 1) initial requests. The two 
tables contrast the solution lengths for a mailbot with k = 3 bags to one which 
can carry k = 8 packages at a time. 



fc = 3 
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# act 
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# act 
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3880 
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814 
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1016 
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5016 
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k = 8 
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580 


21 


3720 


12 


744 


22 


4254 


13 


936 


23 


4836 


14 


1158 


24 


5468 


15 


1412 


25 


6152 


16 


1700 


26 


6890 
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2024 
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7684 


18 


2386 


28 


8536 


19 


2788 


29 


9448 


20 


3232 


30 


10422 



Figure 2.5 gives a detailed analysis of the runtime behavior of the program. 
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Figure 2.5: The computational behavior of the simple program for the mail 
delivery robot in the course of its execution. The horizontal axis depicts the 
degree to which the run is completed while the vertical scale is in seconds per 
100 actions. The graphs are for maximal mail delivery problems with k = 3 
mail bags. 



Depicted are, for three selected problem sizes, the average time for action se- 
lection and state update computation in the course of the execution of the 
program.^ The curves show that the computational effort remains essentially 
constant throughout. A slight general descent is discernible, which is due to 
the fact that the state size decreases in the course of time as fewer and fewer 
requests remain. □ 

Having implemented the simplest possible strategy for the mailbot, there are 
many ways of improving the performance. First, however, we will use our pro- 
gram to illustrate how one can establish important formal properties of FLUX 
agents, such as whether the generated sequence of actions is always possible. 

The results of logic programs are determined by their computation tree. 
Putting aside the question of computational efficiency, the value of an agent 
program is judged by its generated sequence of actions. Most of the interesting 
properties of agent programs are therefore concerned with how the situation 
of the robot unfolds. From this perspective, the crucial nodes in a computation 
tree are those by which actions are executed, that is, queries where the atom 
Execute is resolved. In what follows, by agent program we mean a program 
including the background theory and the kernel Pskemei ■ 

Definition 2.7 An execution node in a computation tree is a query starting 
with atom Execute (a, ti , t 2 ) . 

Consider an agent program P and query Qq , and let T be the computation 
tree for P U Qo}- For each node Q in T, the situation associated 



^ All runtimes in this book were measured on a PC with a 500 MHz processor. 
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with Q is Do([ai, . . . , q;„], S'o) where Execute{ai, L), . . . , Execute{am is 
the ordered sequence of execution nodes to the left and above Q in T (n > 0). 

□ 

It is worth stressing that associated situations are determined by all execu- 
tion nodes that have been resolved, including — if any — those that lie on failed 
branches. This is because executed actions have actually been performed once 
and for all in the outside world, no matter whether backtracking occurs after- 
wards in the agent program. 

Figure 2.6 depicts an excerpt of the computation tree for the mailbot pro- 
gram if applied to initial state (2.2). Starting with the query Main, the first 
branching occurs when resolving the query MainLoop, resulting in four alter- 
native continuations. The leftmost branch eventually fails as no instance of 
Carries {b, 1) holds in the initial state and hence no delivery action is possible. 
The second branch ramifies because multiple instances of Empty (b) hold in 
the initial state, and later since multiple instances of Request{l,r) hold. How- 
ever, all open alternatives are discarded when resolving the cut prior to arriving 
at the first execution node. In Figure 2.6, this derivation step is indicated 
by an arrow pointing downwards. In the depicted excerpt of the tree, situa- 
tion So is associated with all but the bottommost node, which is associated 
with Do{Pickup{l,2), So). 

A property which agent programs should always exhibit is to execute ac- 
tions only if they are possible according to the underlying domain theory. This 
includes the requirement that the first argument of Execute{a, z\, Z 2 ) is fully 
instantiated whenever this atom is resolved, for otherwise the underlying exe- 
cution model of the agent could not actually perform the (partially specified) 
action. The following notion of soundness captures these intuitions. 

Definition 2.8 Agent program P and query Q are sound wrt. a domain 
axiomatization S if for every execution node with atom Execute{a, L) in the 
computation tree for PU Q}, a is ground and S |= Poss{a,a) , where a 
is the situation associated with the node. □ 

The following theorem provides a strategy for writing sound programs. One 
of the conditions is that the underlying domain satisfies the assumptions of The- 
orem 1.17 (page 19), and hence is deterministic. Further aspects are, informally 
speaking, to 

1. use Execute solely in the main derivation, that is, not in a derivation for 
a negated sub-goal; 

2. make sure that no backtracking occurs over execution nodes; 

3. systematically link the execution nodes thus: 

Init{zo), Execute {ai, zq, Zi), Execute{a 2 , Z\, Z 2 ), . . . 

making sure that at the time the execution node is resolved, every ai is 
ground and Poss{ai, Zi-i) holds. 
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Main 



Init{z), MainLoop{z) 



MainLoop{[At{l ), . . .]) 




Poss{Deliver{b), [At{l), 
Execute{Deliver{b), ■ • •], Z2), 

MainLoop{z2) 



Poss{Pickup{b, r), [At{l), 
Execute{Pickup{b, r), [At{l), . . .], Z2), 
MainLoop[z2) 



Holds{At{r), [At{l ), . . .]), 

H olds { Carries {b, r), [j 4 i(l), ...])—!■ 
Execute{Deliver{b), [j 4 t(l), . . .], 2:2), 
MainLoop{z2) 

Holds{Carries{b,l),[-^^{I)j ■ ■ ■]) 

Execute{Deliver(b), . • •], 22), 

MainLoop{z2) 
failure 



H olds {Empty {b), [Af(l), . . .]), 
Holds{At{ri), [At{l ), . . .]) 
Holds{Request{ri,r), [At{l ), . . .]) —> 
Execute{Pickup{b, r), [Af(l), . . .], 22), 
MainLoop{z2) 




Holds{At{ri), [j 4 f(l), . . .]) 
Holds{Request{ri , r) , [At{l), 
Execute{Pickup{l, r), \At{l), . . .], 22) 
MainLoop{z2) 



Holds{Request{l, r), [At{l), 
Execute{Pickup{l,r), \At{l), . . .], 22), 
MainLoop{z2) 

( 1 — ^ ^ 

Execute[Pickup{l, 2 ), [ylJ(l), . . .], 22), 
MainLoop{z2) 



MainLoop{[Carries{l, 2 ), . . .]) 



Figure 2.6: An excerpt of the computation tree for the mailbot program. 
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Formally, we say that an execution node Q 2 starting with Execute{a 2 , Z 2 , -) is 
linked to another execution node Qi starting with Execute{a\^ -,Z\) if Z 2 is 
zid where 9 is the substitution used when resolving Q\. 

Theorem 2.9 Let be a domain axiomatization satisfying the assumptions 
of Theorem 1.17. Let P be an agent program and Qq a query such that ev- 
ery execution node Q in the computation tree for P U {<— Qq} satisfies the 
following: Let Execute{a, z\, Z 2 ) be the leading atom of Q, a the associated 
situation, and Ti a state such that |ti] = Zi, then 

1. Q is in the main tree and there is no branch to the right of Q; 

2. a and z\ are ground while Z 2 is a variable; 

3. if a = So, then E ^ State(So) = t\, else Q is linked to its preceding 
execution node; 

4-. El \= Poss{a, Ti) . 

Then P and Qo a,re sound wrt. S. 

Proof: Consider an execution node Q starting with Execute{a, z\, Z 2 ) and 

whose associated situation is tr = Do{[ai, . . . ,an], So) . By condition 1, all 
execution nodes lie on one branch; hence, unless a = So, Q is linked to an 
execution node whose associated situation is cr„_i = Do{[ai, . . . ,an-i], Sq). 
By induction on n, we prove that S ^ State{a) = t\, which implies the 
claim according to condition 4. The base case n = 0 follows by condition 3. 
In case n > 0, the induction hypothesis along with condition 4 imply that 
S 1= Poss[an,o'n-i). Since S is deterministic according to Theorem 1.17, 
there is a state r such that S ^ State{Do{an,(rn-i)) = t. From the encoding 
of the state update axioms, (2.6), along with Theorem 2.6 and the fact that Q 
is linked to the execution node with associated situation (T„_i it follows that 

S 1= T = Tl . ■ 

Example 1 (continued) With the help of the theorem, our agent program for 
the mailbot can be proved sound wrt. the domain axiomatization of Chapter 1 
for any initial state which satisfies the domain constraints. 

1. There is no negation in the program; hence, all execution nodes are in 
the main tree. Moreover, since every occurrence of Execute{A{x), z, z\) 
is immediately preceded by there are no branches to the right of any 
execution node. 

2. Resolving Init{z) grounds z. Given that 2 : is ground, resolving any 
of P OSS {D diver (b), z), Poss{Pickup{b,r), z) , and ContinueDelivery{d, z) 
grounds the respective first argument. Hence, the first two arguments in 
every execution node are ground. On the other hand, the third argument 
in every occurrence of Execute{a, zi, Z 2 ) remains variable until the atom 
is resolved. 
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3. It is assumed that resolving Init{z) instantiates the argument with a 
FLUX state representing the initial state. This state is used when first 
executing an action. The construction of the clause for MainLoop implies 
that all further execution nodes are linked to their predecessors. 

4. Actions Deliverih) and Pickup{b,r) are verified to be possible prior to 
executing them. According to precondition axiom (1.10) on page 16, ac- 
tion Go{Up) is possible in z if Holds{At{r), z) for some r < n. This is 
indeed the case whenever there is a request from or mail to some ri > r, 
given that the initial state satisfies the domain constraints (1.9) on page 15. 
Likewise, action Go{Down) is possible in z if Holds { At [r), z) for r > 1, 
which is true whenever there is a request from or mail to some ri < r. □ 

It is worth mentioning that all which has been said applies unconditionally 
to programs that continue controlling agents without ever terminating. The 
property of a program to be sound is independent of whether or not it contains 
a finite branch ending with success. Hence, infinite computation trees which 
satisfy the conditions of Theorem 2.9 are guaranteed to generate executable, 
albeit infinite, sequences of actions. 

Our program for the mailbot, however, can be proved to terminate for any 
consistent initial state. Moreover, we can show that upon termination all re- 
quested packages will have been picked up and delivered. 

Proposition 2.10 Let Email be the axiomatization of the mailbot domain, 
including an initial state axiom State{So) = r such that r is consistent wrt. 
the domain constraints in Email ■ Then the computation tree for the program 
Pmaii U {Init{z) <— z = |t] } U Pskernei o,nd the query {<— Main} contains a 
single successful branch. Moreover, if a is the situation associated with the 
success node, then 

Email h ^{^ri,r 2 ) Holds{Request{ri,r 2 ),a) 

A ^(35, r) H olds { Garries {b, r),a) 

Proof: We already know that all actions are possible in the situation in 

which they are executed. By virtue of being finite, the initial state contains 
a finite number of instances of Request and a finite number of instances of 
Garries . Each Pickup action reduces the number of instances of Request which 
hold. Each Deliver action reduces the number of instances of Garries which 
hold while preserving the number of instances of Request. Each Go action 
brings the robot closer to a selected office where it can pick up or deliver a 
package, so that after finitely many Go actions a Pickup or a Deliver fol- 
lows. Consequently, after a finite number of actions a situation a is reached 
in which no instance of Request and no instance of Garries holds. Let r be 
a state such that Email H State{a) = r, then each of Poss{Deliver{b),\T\) , 
Poss{Pickup{b,r),\T\), and GontinueDelivery{d,\T\) fails. Hence, the right- 
most branch of the overall derivation tree ends with resolving MainLoop {\t\) 
to the empty query. ■ 



2.3. CONTROL PROGRAMS 



45 



1 2 3 4 5 6 




Figure 2.7: Carrying two packages with three more waiting to be picked up, the 
mailbot has to choose whether to move up or down the hallway, and if it decides 
to go to room 6 first, then it has to choose which of the two packages to put 
into the last free bag. 



While the simple agent program for the mailbot solves the mail delivery 
problem, it does so rather inefficiently. This should be clear from the observation 
that a robot endowed with more mail bags is not significantly faster, as the 
figures in Section 2.3 indicate. One of the key advantages of high-level agent 
programming is to support concise and modular implementations of complex 
strategies. The mail delivery program, for example, can easily be improved by 
having the robot making better choices when selecting packages to be picked up 
or when heading towards another office. 

Example 1 (continued) Figure 2.7 depicts a typical situation of choice for 
the mailbot. Since there are no packages to be delivered or collected at its 
current location, the robot must decide whether it should move down to room 2 
(to deliver mail) or further up the hallway (to pick up a package). If it goes 
to room 6 first, then there is another choice to be made: Since there is only 
one free bag, the robot has to decide which of the two packages to handle first. 
Obviously, the smart robot decides to walk up to room 6 , grabs and delivers the 
package for 5, goes back to 6 to pick up the second package, and then moves 
down to the other end of the hallway to finish the job. The agent program in 
the previous section, however, makes just arbitrary choices, which most of the 
time are not the optimal ones. 

A more sophisticated strategy is to have the mailbot always take into ac- 
count the distance to its current location when deciding which request to deal 
with next. This concerns both comparing the destination of the packages to 
choose from when filling the mail bag, and calculating the nearest office where 
something is to be done when moving along the hallway. The program depicted 
in Figure 2.8 implements this strategy. As before, packages are delivered in no 
prioritized order, because a specific order would make no difference. However, 
a mail bag is filled with a package only if there is no other package at the cur- 
rent location whose addressee is closer. Likewise, to pick up mail elsewhere, the 
mailbot goes to the nearest office with a request, and it goes to another office 
to deliver a package only if there is no closer office for which it carries mail, too. 
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main_loop(Z) 

poss(deliver(B) , Z) 

fill_a_bag(B, R, Z) 

cont inue_delivery (D , 

true . 



-> execute (deliver (B) , Z, Zl) , 
main_loop(Zl) ; 

-> execute(pickup(B,R) , Z, Zl) , 
main_loop(Zl) ; 

Z) -> execute (go (D) , Z, Zl) , 
main_loop(Zl) ; 



fill_a_bag(B, R1 , Z) 

poss (pickup (B, _) , Z) -> 
holds (at (R) , Z) , 
holds (request (R,R1) ) Z) , 

\+ ( holds (request (R,R2) , Z) , closer(R2, R1 , R) ). 



continue_delivery (D, Z) 
holds (at (R), Z) , 

( holds (empty (_) , Z) , 
holds (request (_, _) , Z) 

-> ( holds (request (R1 ,_) , Z) , 

\+ ( holds (request (R2 ,_) , Z) , closer (R2, R1 , R)) ) 

; ( holds(carries(_,Rl) ) Z) , 

\+ ( holds (carries (_ ,R2) , Z) , closer(R2, Rl, R)) ) 

), 

( R < Rl -> D = up 

; D = down ) . 



closer (Rl, R2, R) abs(Rl-R) < abs(R2-R). 



Figure 2.8: A smart mailbot agent in FLUX. 
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The improved FLUX program is sound, too (see Exercise 2.4). Running this 
program with the initial state of Figure 1.1, the mailbot needs 74 instead of 80 
actions to carry out all requests. The following tables show the lengths of the 
action sequences applied to the same set of maximal mail delivery problems as 
in Section 2.3. 



k = 3 



n 


# act 


n 


# act 


9 


264 


20 


2020 


10 


344 


21 


2274 


11 


430 


22 


2600 


12 


534 


23 


2930 


13 


656 


24 


3254 


14 


798 


25 


3676 


15 


956 


26 


4098 


16 


1100 


27 


4470 


17 


1286 


28 


4996 


18 


1516 


29 


5478 


19 


1760 


30 


5980 



k = 8 



n 


# act 


n 


# act 


9 


206 


20 


1350 


10 


272 


21 


1498 


11 


342 


22 


1686 


12 


416 


23 


1890 


13 


502 


24 


2098 


14 


592 


25 


2320 


15 


694 


26 


2546 


16 


794 


27 


2792 


17 


908 


28 


3054 


18 


1054 


29 


3306 


19 


1180 


30 


3590 



A comparison with the results for the naive program shows that for fc = 3 
the improved strategy leads to solutions which are almost just half in length. 
Endowing the robot with five more slots reduces the solution length up to an 
additional 40%. 

A comparative analysis of the runtime behavior of the two agent programs 
brings to light an interesting phenomenon. Not unexpectedly, the improved 
algorithm solves each individual problem much faster, as can be seen by the 
following figures (in seconds CPU time) for a robot with fc = 3 mail bags: 



simple strategy improved strategy 



n 


time 


n 


time 




n 


time 


n 


time 


11 


0.21 


21 


5.23 




11 


0.15 


21 


3.04 


12 


0.32 


22 


6.57 




12 


0.21 


22 


3.79 


13 


0.45 


23 


8.21 




13 


0.31 


23 


4.59 


14 


0.68 


24 


10.18 




14 


0.43 


24 


5.63 


15 


0.95 


25 


12.50 




15 


0.60 


25 


6.91 


16 


1.31 


26 


15.14 




16 


0.80 


26 


8.32 


17 


1.83 


27 


18.15 




17 


1.09 


27 


9.92 


18 


2.42 


28 


21.99 




18 


1.44 


28 


11.93 


19 


3.14 


29 


26.26 




19 


1.88 


29 


14.09 


20 


4.04 


30 


31.14 




20 


2.38 


30 


16.51 



On the other hand, a closer examination reveals that the time for action 
selection and update computation tends to be higher for the second program. 
This can be seen by comparing the initial segments of the graphs in Figure 2.9 
with those of Figure 2.5. In particular with the graph for n = 30 it is easy to see 
that at the beginning more time is needed, on average, to select an action. This 



48 



CHAPTER 2. SPECIAL FLUX 




Figure 2.9: The computational behavior of the improved program for the mail 
delivery robot in the course of its execution. The horizontal axis depicts the 
degree to which the run is completed while the vertical scale is in seconds per 
100 actions. The graphs are for maximal mail delivery problems with k = 3 
mail bags. 



is not surprising since the sophisticated strategy requires to compare several 
instances before making a decision, as opposed to just making an arbitrary 
choice. The general principle behind this observation is that improving the 
quality of the generated action sequence is usually payed for by an increased 
computation time per action. In the extreme case, there might be too much of 
a tradeoff in order for a highly sophisticated action selection strategy to be of 
practical value (see also Exercise 2.6). 

In our particular example, however, the smart robot turns out to be even 
faster on average in the long run: The graphs in Figure 2.9 show a steeper 
general descent as the program proceeds. This is because the state size decreases 
faster due to generally shorter delivery times. As a consequence, the average 
computation time for the problem with n = 30 offices, say, is 31.14/10672 = 
2.92 • 10“^ seconds per action under the simple strategy and just 16.51/5980 = 
2.76 • 10“^ if we employ the sophisticated program. □ 

2.4 Exogenous Actions 

The programs for the mailbot have been written under the assumption that all 
delivery requests are given initially and that the work is done once all requests 
have been carried out. In a practical setting, however, it should be possible 
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Perform{a, e) 




Physical Agent 



Figure 2.10: Architecture of FLUX programs for exogenous actions. 



at any time to dynamically issue new requests (or, for that matter, to cancel 
a request). The automatic post boy must then be on the alert at any time 
and react sensibly to changes as soon as they occur. Most agents are actually 
embedded in an environment which includes other active entities, be it humans 
or fellow agents. As a consequence, some state properties may not be under 
the sole control of one agent. Agents must therefore take into account actions 
besides their own when maintaining their internal world model. 

From the subjective perspective of an agent, an action is exogenous if it 
is not performed by the agent itself but does affect one or more of the relevant 
fluents. In the presence of exogenous actions, the interaction between an agent 
and its environment is no longer unidirectional. Figure 2.10 depicts the modified 
architecture of FLUX agent programs in which exogenous actions are accounted 
for. The standard FLUX predicate Perform{a) is extended into Perform{a,e), 
where e is a (possibly empty) list of actions that happen as the agent performs 
action a. These exogenous actions are taken into account upon updating the 
internal world model. This is achieved by an extension of the kernel predicate 
Execute{a, zi, Z 2 ) as shown in Figure 2.11. According to this definition, a suc- 
cessor state is obtained by inferring the effect of the agent’s action a and of all 
exogenous actions that have additionally occurred. So doing requires the agent 
to be equipped with state update axioms also for the possible exogenous actions 
in a domain. 

Example 1 (continued) The mail delivery scenario shall be extended by the 
exogenous actions of adding and cancelling requests. Furthermore, we introduce 
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state_updates(Z, [] , Z) . 
state_updates(Zl , [A|S], Z2) 

state_update (Z1 , A, Z) , state_updates(Z, S, Z2) . 

execute (A, Zl, Z2) 

perform(A, E) , state_updates (Zl , [A|E], Z2) . 



Figure 2.11: Execution in the presence of exogenous actions. 



the simple action of the mailbot waiting for the arrival of new requests in case 
there is none at the moment: 

Add : N X N 1-^ action Add(ri,r 2 ) = new request from ri to T 2 
Cancel: N x N action Cancel{ri,r 2 ) = request ri to C 2 cancelled 

Idle : ACTION 

We assume that new requests can only be issued if the arguments differ and 
are in the allowed range. Requests can be cancelled just in case they exist, so 
that it is too late for cancellation if the robot has already picked up the package 
in question. Idling is, of course, always possible (though it should not be the 
preferred action of the robot). Hence the following precondition axioms: 



Poss{Add{ri,r 2 ), z) = r\ ^ T 2 A 1 < ri, t ’2 < n 
Poss{Cancel{ri,r 2 ), z) = Holds{Request{ri,r 2 ), z) 

Poss{Idle,z) = T 

The state update axioms for the new actions are straightforward: 
Poss{Add{ri,r 2 ), s) D 

State{Do{Add{ri,r 2 ), s)) = State(s) + Request{ri,r 2 ) 
Poss{Cancel{ri,r 2 ), s) D 

State{Do{Cancel{ri,r 2 ), s)) = State{s) — Request {ri,r 2 ) 

Poss{Idle,s) D State{Do{Idle, s)) = State{s) 

Figure 2.12 shows the FLUX encoding of these state update axioms.^ Based 
on this extended background theory, our FLUX programs for the mailbot can 
be directly applied to the new setting, with just a simple modification by which 
the robot is triggered to idle in case there is nothing to do; see Figure 2.12. The 
extended control program enables the robot to react immediately and sensibly 
whenever requests are added or cancelled. A sample scenario is illustrated in 
Figure 2.13, whereby it is assumed that the robot employs the improved selection 

® Exogenous actions are necessarily possible when they happen. Hence, there is no need to 
encode the underlying precondition axioms. These will be needed, however, for stating 
and proving correctness of FLUX programs in the presence of exogenous actions. 
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state_update (Z1 , add(Rl,R2), Z2) 

update(Zl, [request (R1 ,R2)] , [] , Z2) . 

state_update (Z1 , cancel (Rl, R2) , Z2) 

updateCZl, [] , [request(Rl,R2)] , Z2) . 

state_update (Z , idle, Z) . 

main_loop(Z) 

( poss (deliver (B) , Z) -> execute (deliver (B) , Z, Zl) ; 

f ill_a_bag(B, R, Z) -> execute(pickup(B,R) , Z, Zl) ; 

continue_delivery (D , Z) -> execute (go (D) , Z, Zl) ; 
executeddle, Z, Zl) 

), 

main_loop(Zl) . 



Figure 2.12: Additional state update axioms and slightly modified main loop 
of a mailbot controller which allows for adding and cancelling requests during 
program execution. 



strategy: After picking up the initial requests in rooms 1 and 2, the robot 
heads towards office 4 with the intention to collect a third package. However, 
the corresponding request is cancelled while the robot goes from room 3 to 4. 
At the same time, a new request arises in office 3. This causes the robot to 
turn around immediately and go back to this room. In the meantime, a second 
request from office 6 arises. At the end, after dropping the package that came 
from office 6 the robot idles in front of room 2 awaiting further exogenous 
actions by which new requests are issued. □ 

Exogenous actions can be viewed as input to a FLUX program. This input 
has obviously significant influence on the actual computation. In particular, 
the generated sequence of actions depends on the exogenous actions that occur 
during the run of a program. As a consequence, it is no longer appropriate to 
speak of the computation tree of a program where there can be many different 
ones. As the computation trees for a program are each determined by the 
input, they can be characterized by the substitutions for e in every resolved 
Perform{a,e) node: 

Definition 2.11 Let T be a computation tree for an agent program and 
query. Let Execute{ai, I), Execute{a 2 , . be the ordered sequence of 
execution nodes in T. Tree T is said to process exogenous actions £i,S 2 , - ■ ■ 
if Perform{ai, ei),Perfornn{a 2 , 62), . . . is the ordered sequence of child nodes of 
the execution nodes in T and {ei/?i}, {62/^2 }, . . . are the substitutions used 
to resolve these child nodes. Tree T is said to generate the sequence of actions 
oi, 02, ■ ■ n 
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1 2 3 4 5 6 




1 2 3 4 5 6 




1 2 3 4 5 6 




1 2 3 4 5 6 




Figure 2.13: Sample run where the requests change dynamically. 



2.4. EXOGENOUS ACTIONS 



53 



As an example, Figure 2.14 depicts skeletons of two different computation trees 
for the mailbot program. It is assumed that the initial state contains no requests 
and all mail bags are empty. The tree sketched on the left hand side processes 
the exogenous actions 



[], [Add{l,2)], [], [], [], [], ... 

and generates actions Idle, Idle, Pickup{l,2), Go{Up) , Deliver{\) , Idle, . . . The 
tree on the right hand side processes the exogenous actions 

[Add{2,l)], [], [], [], [], [], ... 

and generates Idle, Go{ Up), Pickup{l, 1), Go(Down), Deliver)!), Idle, . . . 

The exogenous actions along with the actions of the agent determine the 
situation the agent is in at each stage of the program. This is crucial when it 
comes to proving that a program is sound, which requires all executed actions 
to be possible. Suppose, for example, the mailbot starts in a state without any 
initial request and eventually executes a Pickup action. This action can only 
be justified by a preceding exogenous action by which a new request has been 
issued. Thus it is necessary to generalize the notion of an associated situation 
as follows: 

Definition 2.12 Consider an agent program P and query Qq, and let 
T be a computation tree for P U Qo}- Let Q be a node in T and 
Execute{ai, Execute{an, -) the ordered sequence of execution nodes 

to the left and above Q In. T [n > 0). If £\,£ 2 , ■ ■ ■ ,Un, . . . are the ex- 
ogenous actions processed by T, then the situation associated with Q is 
Do{[ai,£i,. . . ,an,£n],So).‘^ □ 

It is easy to see that this definition coincides with the previous one in the 
special case that no exogenous action at all happens. For example, in both 
trees of Figure 2.14 the topmost and the second node are associated with Sq, 
and, say, the sixth node in the tree on the left hand side is associated with 
Do)\Idle, Idle, Add{!,2)], Sq) whereas the sixth node in the tree on the right 
hand side is associated with Do[[Idle, Add{2,!), Go{Up)],Sq). 

We are now in a position to generalize to exogenous actions the notion 
of soundness of programs. Informally speaking, an agent program is sound 
if for any sequence of exogenous actions, all generated actions are possible in 
the situation in which they are executed — provided that the exogenous actions 
themselves are possible. The latter restriction allows to ignore impossible inputs, 
for which a program need not be prepared. 

Definition 2.13 Agent program P and query Q are sound wrt. a domain 
axiomatization S if for every computation tree T for PU{<— Q} the following 

^ By a slight abuse of notation, we identify a list Si with a (possibly empty) sequence of 
actions. 
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Main Main 



MainLoop{[At{l ), . . .]) 


MainLoop{[At{l ), . . .]) 


Perform{Idle, ei) 
|{ei/[]} 

MainLoop{[At{l ), . . .]) 


Perform{Idle, ei) 

{ei/[Add{2,l)]} 

MainLoop{[Request{2, 1), At {\), . . .]) 


Perform{Idle, € 2 ) 

\{e 2 /[Add(l, 2 )]} 

MainLoop{[Request{l, 2), At{l ), . . .]) 


Perform{Go{ Up), 62 ) 

|W[]} 

MainLoop{[Request{2, 1), At{2 ), . . .]) 


P erf orm {Pickup {1, 2), es) 

|W[]} 

MainLoop{[Carries{l, 2), At{l ), . . .]) 


Perform{Pickup{l, 1), 63 ) 

1 {e3/[]} 

MainLoop {[Carries {1, 1), At{2 ), . . .]) 


Perform{ Go{ Up), 64 ) 

1 W[]} 

MainLoop {[Carries {1, 2), At{2 ), . . .]) 


Perform{Go{Down), 64 ) 

1 {e4/[]} 

MainLoop{[Carries{\, 1), At {\), . . .]) 


Perform{Deliver{l) , 65 ) 

|W[]} 

MainLoop {[At {2 ), . . .]) 


Perform{Deliver{l) , 65 ) 

1 {65/[]} 

MainLoop{[At{l ), . . .]) 


Perform{Idle, cq) 

IW[]} 


Perform{Idle, cq) 
1 {e6/[]} 



Figure 2.14: Two different computation trees for the mailbot program with 
exogenous actions. 
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holds: Let £i,£ 2 , ■ . ■ be the exogenous actions processed by T and Oi, q; 2 j • • ■ 
be the actions generated by T. If for all i = 1,2, . . ., 

E ^ Poss{£^, Do{a^,a^)) 

then for all i = 1,2, . . ., 

S ^ Poss{ai,ai) 

where (Ji is the situation associated with the i-th execution node in T . □ 

That is to say, a program is sound if in every computation tree in which all 
exogenous actions are possible, all generated actions are possible, too. 

Theorem 2.9, which provides a general strategy for writing sound programs, 
carries over to FLUX with exogenous actions (see Exercise 2.10). This is so be- 
cause the definition of execution of Figure 2.11 ensures that the internal world 
model is updated according to the actions of both the agent and the environ- 
ment. The FLUX programs for our mailbot can thus be proved sound also 
in case requests are dynamically added or cancelled (see Exercise 2.11). The 
possibility to issue requests at runtime raises, however, a novel challenge. If 
constantly new requests occur, then both of our control programs risk to never 
handle some requests if the robot is all the time busy with new ones. For practi- 
cal purposes it is therefore necessary to employ a refined selection strategy that 
guarantees that every request is handled eventually. Exercise 2.12 is concerned 
with developing such a strategy. 



2.5 Bibliographical Notes 

The use of logic as a programming paradigm has been introduced in [Kowal- 
ski, 1974]. Around the same time, the logic programming language Prolog 
(for: Programmation en lo^ique) has been developed [Colmerauer et al., 1972]. 
There are many textbooks on the theory of logic programming, among which are 
the classical [Lloyd, 1987] and the more recent [Apt, 1997]. Prolog companions 
such as [Clocksin and Mellish, 1994] focus on the practical side of programming. 
The negation-as-failure rule and the completion semantics have been developed 
in [Clark, 1978]. Our definition of computation trees with negation and cut 
follows [Apt and Bol, 1994; Apt, 1997]. 

General-purpose languages and systems for reasoning agents and robots have 
not been developed until recently, beginning with GOLOG (for: Algol in 
logic) [Levesque et al, 1997]. The underlying action formalism is situation 
calculus. A typical encoding in GOLOG of how a fluent changes upon the 
performance of actions is 

holdsCat (R) ,do(A,S) ) :- A=go(D), holds (at (Rl) , S) , 

(D=up, R is Rl+1 ; D=down, R is Rl-1) 



\+ A=go(_), holds (at (R) , S) . 
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Put in words, either At(r) becomes true by performing a Go action with the 
corresponding At{ri) being true in the preceding situation, or the location of 
the mailbot remains unchanged because a different action is performed. The 
logic programs for GOLOG described in the literature all apply the principle 
of regression to evaluate conditions in agent programs: The question whether, 
for example, At{r) holds after a sequence of actions is solved by recursively 
applying the clause just mentioned until the initial situation is reached. A 
consequence of this is that the computational effort to evaluate a condition in- 
creases with the number of actions that have been performed. Gontrol strategies 
are encoded in GOLOG programs using a special syntax for control structures 
such as conditional branching and loops. GOLOG has been extended into var- 
ious directions, such as the incorporation of sensing actions, concurrency, or 
decision theoretic elements [Giacomo and Levesque, 1999; Lakemeyer, 1999; 
Giacomo and Levesque, 2000; Boutilier et ai, 2000; Grosskreutz and Lake- 
meyer, 2000]. The textbook [Reiter, 2001a] provides a detailed account of the 
mathematical foundations of GOLOG. 

FLUX as a high-level programming language and system has been introduced 
in [Thielscher, 2002a]. The computational behaviors of GOLOG and FLUX 
have been compared in [Thielscher, 2004], where it is shown how the principle 
of progression allows to evaluate conditions directly against the updated (i.e., 
progressed) world model. Other systems for automated reasoning about actions 
are described in [Shanahan and Witkowski, 2000; Kvarnstrom and Doherty, 
2000; Giunchiglia et al, 2004]. 
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2.6 Exercises 

2.1. Formulate queries to the FLUX background theory for the mailbot domain 
to compute the following: 

(a) the initial requests with the farthest distance between sender and 
recipient; 

(b) the initially possible actions; 

(c) an executable sequence consisting of five actions which ends with a 
delivery in office 4. 

2.2. Prove Lemma 2.5. 

2.3. (a) Extend the FLUX kernel of Figure 2.2 by clauses P equal defining 

a predicate Equal{z\, Z 2 ) with the intended meaning that the two 
states z\ and Z 2 are equal under the semantics of fluent calculus. 

(b) Construct the completion of the program thus extended. Prove that 

COMP[Fskemel^ P equal] h Equal {{ tiI.Itz}) iff S sstate 1^ — '^2 

for any two ground states ri , T 2 and any of their FLUX encodings 

[n], M- 

(c) Use the new predicate to generalize the definition of Update so as to 
allow for verifying that two ground states satisfy an update equation. 

2.4. Show that the improved agent program of Figure 2.8 for the mailbot is 
sound. 

2.5. Find and implement a refined strategy for the mailbot by which the so- 
lution lengths for maximal mail delivery problems are further optimized 
in comparison with the program of Figure 2.8. Show that the program is 
sound wrt. the axiomatization for the mail delivery domain. 

2.6. Write a program for the mailbot which always generates the optimal solu- 
tion. Compare its runtime behavior with that of the programs of Figure 2.8 
and of the previous exercise. 

2.7. Implement Exercise 1.6: 

(a) Modify the FLUX encoding of the background theory for the mail 
delivery world by the 3-place Request predicate and the Putdown(h) 
action. 

(b) Find and implement a strategy to reduce the number of Go actions 
using the new action. Show that the program is sound wrt. the 
domain axiomatization developed in Exercise 1.6. 

2.8. Implement the domain of Exercise 1.7 and write an agent program with the 
aim to minimize the number of transfer actions. Show that the program 
is sound wrt. your domain axiomatization. 
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2.9. Consider a robot working in an office building with two elevators Ei and 
E 2 next to each other. At each floor there is a single call button, which 
can be activated if no elevator happens to be at this floor. The activation 
has the nondeterministic effect that either of the two elevators arrives. 

(a) Axiomatize this domain using the fluents At (a;, n), representing that 
X € {El, E 2 , Robot} is at floor n € N, and In(x), representing 
that the robot is inside of elevator x. The actions shall be Call{n), 
pressing the call button at floor n, and Enter (x), entering the ele- 
vator X. Suppose that the effect of calling an elevator be specified 
by this nondeterministic state update axiom: 

Poss{Call{n), s) D 

(3n') ( Holds{At{Ei,n'), s) A 
State {Do{C allin), s)) = 

State(s) — At{Ei,n') + At{Ei,n) ) 

V 

(3n') ( Holds[At{E 2 ,n'), s) A 
State {Do{C allin), s)) = 

State{s) — At{E 2 ,n') + At{E 2 , n ) ) 

Define suitable precondition axioms, a state update axiom for Enter, 
and appropriate domain constraints. 

(b) Encode the axioms in FLUX. Show that there is an unsound agent 
program which satisfies all conditions of Theorem 2.9 except that the 
underlying domain is not deterministic. 

2.10. Prove Theorem 2.9 in the presence of exogenous actions and with the 
generalized notion of soundness of Definition 2.13. 

2.11. Prove soundness of the mailbot program of Section 2.4, which allows for 
dynamically adding and cancelling requests. 

2.12. (a) With regard to the mailbot program for exogenous actions, find an 

initial state and an (infinite) sequence of exogenous Add actions such 
that one of the initial requests is never handled. 

(b) A general solution to this problem is to give requests increasing pri- 
ority the longer they have been around. One way of implementing 
this is to extend the mailbot domain by a fluent Time(t) such that 
t indicates the number of Go actions the robot has performed since 
the beginning. Redefine, to this end, the fluents Request (ri,r 2 ) and 
Carries{b,r) so as to include the information at what time the re- 
quest in question has been issued. Modify the state update axioms 
accordingly. Program a refined control strategy by which every re- 
quest is guaranteed to be carried out eventually. (Take care also that 
the robot does not carry around some package forever!) 



Chapter 3 



General Fluent Calculus 



Imagine a cleaning robot in an office building whose task is to empty all waste 
bins of a floor. As this is a rather noisy procedure, the “cleanbot” works after 
hours. Still, it is supposed to not burst into any office which is occupied by 
people staying in late. At the beginning of its route, the robot does not know 
which offices are still in use. Fortunately, the cleanbot is equipped with a 
light sensor which is activated whenever inside of or adjacent to a room that is 
occupied. However, the sensor does not enable the robot to tell which direction 
the light comes from. Figure 3.1 depicts a sample office layout and scenario.^ 
The elementary actions of the robot are to go forward to the adjacent square, 
to turn clockwise by 90 degrees, and to empty the waste bin at the current 
position. The challenge is to write a control program by which the cleanbot 
empties as many bins as possible without the risk to disturb anyone. 

The essential difference to the mailbot of Chapter 1 is that the cleanbot 
acts under uncertainty as it does not have complete state knowledge. Its sensor 
allows the robot to acquire information in the course of time, yet there is no 
guarantee that it ever attains the full picture. Uncertainty requires the robot 
to be cautious: If it does not know whether a certain office is free, it should 
not enter. Consider, for example, the situation of the robot of Figure 3.1 after 
having successively emptied the waste bins at (1,1), (1,2), and (1,3). There, 
it senses light. Since it cannot be decided whether the light comes from office 
(1, 4) or maybe (2, 3), the robot should avoid both of them for the moment. So 
it goes back and continues with cleaning the bin in the hallway at location (2,2). 
Sensing no light there, it follows that after all office (2,3) cannot be occupied. 
Moreover, if the cleanbot is smart enough to recall that it already knew that 
one of (1,4) or (2,3) must be occupied, then it can now indirectly conclude 
the state of the remote office (1,4). This conclusion relies on the ability to 
interpret and logically combine sensor information that has been acquired over 
time. 

This chapter and the following two are devoted to the question of how to 

^ For the moment, it is assumed that people do not leave while the robot performs its 

cleaning task. A dynamic variant of this domain is considered in Chapter 7. 
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Figure 3.1: Layout of an office floor. It is assumed that waste bins are in every 
office and every square of the corridor. Room (1,1) is the home of the cleanbot. 
In the scenario depicted in the left diagram, four offices are still in use. The 
diagram on the right hand side shows the locations in which a light sensor would 
be activated that evening. 



program autonomous agents that are capable of reasoning and acting with in- 
complete information. We begin by introducing the general fluent calculus, 
which allows to 

3.1. specify and reason with incomplete states; 

3.2. update incomplete states. 



3.1 Incomplete States 



General fluent calculus is a theory of reasoning about actions in the presence 
of incomplete states. The syntax of the general theory is that of special fluent 
calculus, whose signature can be readily used to formulate partial state knowl- 
edge. 



Example 2 The relevant state information for the cleaning robot are its 
position, the direction it faces, whether a location is occupied, and whether the 
waste bin at a location is cleaned. Accordingly, we will describe the states by 
these four fluents: 



At 

Facing 

Occupied 

Cleaned 



N X N 1-^ FLUENT 
N I— > FLUENT 
N X N 1-^ FLUENT 
N X N 1-^ FLUENT 



At{x,y) = robot is at square (x, y) 
Facing (d) = robot faces direction d 
Occupied (x,y) = square (x,y) occupied 
Cleaned{x,y) = bin at (x,y) cleaned 



The four possible directions shall be encoded by the numbers 1 (north), 2 
(east), 3 (south), and 4 (west), respectively. As usual, we introduce an axiom 
of uniqueness of names, here: UNA[At, Facing, Occupied, Cleaned], 
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Suppose that initially the robot faces north, then we may say that the robot 
knows the following facts about the initial situation: 

Holds{At{l,l)^ So) A Holds{Facing{l), So) A 
-^Holds{Occupied{l,l), So) /\ 

-^Holds\Occupied\2,T), So) A.. .A -^Holds{Occupied{I,I>), So) A 
(Vx, y) ~^H olds {Cleaned {x, y), So) 

That is, given are the robot’s location and orientation, the fact that neither the 
cleaning room nor any of the squares in the corridor is occupied, and that no 
waste bin has been cleaned thus far. This is an example of an incomplete state 
specification. Instead of equating the expression State{So) with a ground term, 
formula (3.1) merely puts some constraints on this state (in the disguise of the 
Holds macro). 

Further information about the initial state can be derived from the general 
domain constraints that apply to the cleaning robot world: 

(3a;, y) {Holds{At{x, y),s) A 1 < x,y < 5) 

Holds{At{xi,yi), s) A Holds{At{x 2 ,y 2 ), s) D x\=X 2 Ayi=y 2 
(3d) {Holds {Facing {d) ^ s) A 1 < d < 4) 21 

Holds{Facing{di),s) A Holds{Facing{d 2 ),s) D di = d 2 \ ■ / 

{'^x) ^Holds{Occupied{x,0), s) A {\/x) ^Holds{Occupied{x,6), s) 
iyy) ^Holds{Occupied{Q,y), s) A {\/y) ^Holds{Occupied{6^y), s) 



Put in words, in any situation the robot is somewhere unique (first and second 
constraint) facing a unique direction (third and fourth constraint), and offices 
outside of the boundaries of the floor cannot be occupied (fifth and sixth con- 
straint). 

When it comes to logical reasoning, there is no essential difference between 
formulas which merely constrain states and those that fully specify a state. 
Consider, for instance, the following formula, which is central to the cleanbot’s 
understanding of its environment. It defines the meaning of light at a certain 
location {x,y) in a particular state z: 

Light{x,y,z) = 

Holds{Occupied{x,y), z) \/ i"? 

Holds{Occupied{x + l,y), z) V Holds {Occupied {x,y + 1), z) V ' ' 

Holds {Occupied {x — 1, y), z) V Holds {Occupied {x, y — 1), z) 



Now, for the sake of argument suppose it is given that initially there is no light 
at square (1,2) but at square (1,3), that is. 



^Light{l,2, State{So)) A Light{l,3, State{So)) 
From axiom (3.3) and -^Light{l,2, State{So)) it follows 



Holds{Occupied{l, 3), So) 



(3.4) 
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On the other hand, from Light(l^ 3, State(So)) it follows 



Holds{Occupied{l,3), So) V 

Holds{Occupied{2,3), So) V Holds{Occupied{l, A) , So) V 
Holds{Occupied{0,3i), So) V Holds{Occupied{l,2), So) 



The penultimate domain constraint in (3.2) implies that there is no room with 
coordinates (0, 3) which can be occupied, nor can the hallway square (1, 2) be 
occupied according to the initial specification, axiom (3.1); hence, with (3.4) it 
follows 



Holds{Occupied{2,3), So) V Holds{Occupied{l,4:), So) 



This disjunction cannot be resolved further. If it were additionally given that 
Light {2, 2, State (So)), then by (3.3), ^Holds{Occupied{2,3), So), which im- 
plies Holds{Occupied{l,A), So) ■ □ 



From a semantic point of view, a crucial difference between complete and 
incomplete states is that the former, according to Definition 1.4, always contain 
finitely many fluents. In contrast, it is straightforward to specify an incomplete 
state in which an infinite number of fluents hold. A simple example is the 
formula 



(Vn) Holds{F{n), So) 



where F : N fluent. The fact that states can be infinite is, however, not 
adequately captured by the foundational axioms of special fluent calculus. A 
particular deficiency arises with the notion of state difference, which is central to 
fluent calculus when it comes to the representation of actions and their effects. 
The problem is that the following formula is satisfiable under the axiomatic 
foundation of special fluent calculus: 



(3z,/)(VzW^-/ 



That is to say, there are (infinite) states z for which fluent removal is not defined 
(see Exercise 3.1). As a consequence, the axiomatic foundation of special fluent 
calculus is too weak to define update for actions with negative effects in the 
presence of incomplete states. 

In order to guarantee that state difference is always defined, an additional 
foundational axiom stipulates the existence of a state for every combination 
of fluents. Since first-order logic is too weak for this purpose, general fluent 
calculus appeals to second-order logic. 
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Second-order logic adds sorted variables for predicates and functions to the 
language of first-order logic. Interpretations for second-order formulas assign 
relations (of the right arity and sort) to predicate variables, and mappings 
(of the right arity and sort) to function variables. The second-order formula 
{\/P){3x) P{x), for example, is contradictory, because there exists an assign- 
ment for variable P, viz. the empty relation, which is false for any x. Substi- 
tutions for predicate and function variables in formulas use A-expressions. 
These are of the form Xxi, . . . ,Xn-T with n being the arity of the variable 
and where r is a first-order formula or term, respectively. The result of the 
substitution is that the variable expression P{ti, . . . ,tn) (or /(ti, . . . , tn), 
respectively) is replaced by r{a;i/ti, . . . ,Xn/tn\. E.g., applying the substitu- 
tion {P/Xd. Facing (d) A ^Facing (d)} to the formula (3x) P(x) results in the 
(inconsistent) first-order formula (3a;) {Facing{x) A -^Facing{x)) . 



Definition 3.1 The foundational axioms Estate of fluent calculus are 

Esstate augmented with the axiom of existence of states: 

(VP)(3z)(V/) {Holds if, z) = P{f)) (3.5) 

where P is a second-order predicate variable of sort fluent. □ 

The new axiom is very powerful. It implies the existence of a state for any 
intensional specification one might think of. If, for example, we wish to construct 
a state in which the fluent P : N ^ fluent holds just for all multiples of 7 
between 14 and 98, and the fluent G : N fluent holds for all even numbers 
and nothing else holds, then the instance of (3.5), 

{P/A/. (3n : N) (/ = P(7 • n) A 1< n< 15) V (3n : N) / = G(2 • n)} 

guarantees that such a state exists. 

The axiom of state existence can be safely added, unlike its counterpart 
in general set theory: Stipulating the existence of a set for any intensional 
definition gives rise to the infamous Russell paradox of the set of all sets which 
do not contain themselves. No such contradiction exists for Estate since the 
second-order variable in axiom (3.5) ranges over fluents only. If P were replaced 
by the analogue to the paradox, that is, the expression Xf.^Holds{f,f), then 
the axiom just stipulates the existence of the empty state. Thus the axiomatic 
foundation of general fluent calculus is consistent, too. 

Proposition 3.2 Estate is consistent. 

Proof: Let t be an interpretation whose domain includes all sets of ground 

fluents and such that 



0 ‘ = {}; 
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• /‘ = {/} for each ground fluent /; 

• {Zl ° Z 2 )" = Z^U Z2- 

Then t is a model for Egstate (of- the proof of Proposition 1.3 on page 7). It 
is also a model for (3.5): Let P‘‘ be any unary relation of ground fluents. If 
we define Z = {f : (/) e P''}, then for any / there exists a set Z' satisfying 
Z = {/} U Z' just in case (/) G PL ■ 

The following result shows that under the axiom of state existence, for any 
state z any fluent /, there exists a state which is as z but without /. This 
implies that the definition of fluent removal extends to infinite states in general 
fluent calculus. Incidentally, the proof of this proposition illustrates how the 
second-order axiom is used in logical derivations. 

Proposition 3.3 (Rewrite Law) 

Sstate h Holds{g,z) = {3z’) {z = g o z' f\^ Holds {g,z’)) 

Proof: Suppose Holds{g,z). Consider the instance 

{P/A/. Holds{f, z) A / / g} 

of the axiom of state existence. This instance implies that there is a state, call 
it z', such that 



(V/) {Holds{f,z') = Holds{f,z) Af^g) (3.6) 

From (V/) {Holds{f, z') A f ^ g) it follows that ~^Holds{g, z'). Decomposition 
and irreducibility imply Holds{f,g o z') = [/ = 5 V Holds{f , z')]] hence, ac- 
cording to (3.6), Holds{f,goz') = [f = g V Holds{f, z)]. From the assumption 
Holds{g,z) it follows that [f = gV Holds{f,z)] = Holds{f,z). Consequently, 
Holds{f,goz') = Holds{f,z). The axiom of state equivalence implies that 
g o z' = z. 

For the converse, suppose (3z') z = g o z' , then Holds{g,z). m 

As we will see in the following section. Proposition 3.3 lays the foundation for 
inferring the update of incomplete states. Therefore, the axiom of state existence 
is indispensable in general fluent calculus. 



3.2 Updating Incomplete States 

Agents with partial state knowledge face the challenge to maintain a world 
model which remains incomplete throughout the execution of the program. The 
concept of a state update axiom as a way to formalize the effects of actions 
applies in the presence of incomplete states, too. However, in special fluent cal- 
culus the result of an action is easily inferred by instantiating an update equation 
with the term that describes the current state, followed by some straightforward 
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rewriting steps. This inference rule is not directly applicable to incomplete state 
specifications such as (3.1), because there is no explicit compound state term 
which can be readily inserted into the update equation at hand. 

Proposition 3.3 helps here. It suggests a (fully mechanic) way of combining 
pieces of state information into one equation, which can then be used in combi- 
nation with an update equation. More precisely, the goal of this procedure is to 
obtain from an arbitrary incomplete state specification for a situation s a sin- 
gle equation State{s) = r, possibly accompanied by further axioms constraining 
the variables in r. 

Example 2 (continued) To begin with, the following is obviously true: 

(3z) State(So) = z 

Next, we integrate into this equation all pieces of information given in state 
specification (3.1). Consider the first atom, Holds{At{l,l),So). Combining 
this with the above equation implies 

(3z) {State(So) = z A Holds{At{l, 1), z)) 

According to Proposition 3.3, this can be equivalently written as 

(3z') ( State(So) = At{l, 1) o z' A ^Holds{At{l, 1), z') ) (3.7) 

Simple macro expansion of Holds{At{l, 1), So) would of course yield the same 
equation for State(So). However, the additional requirement that z' does not 
contain At (1,1) will prove helpful later when it comes to inferring negative 
effects of actions. 

Continuing with the atom Holds{Facing(l), Sq) in (3.1), from (3.7) it follows 

(3z') ( State(So) = At(l, 1) o z' A Holds {At{l, 1), z') , , 

A Holds{Facing{l), At(l,l) ° z')) 

The conjunct in the second line is equivalent to Holds {Facing (1), z') accord- 
ing to the foundational axioms of decomposition and irreducibility along with 
uniqueness-of-names. Hence, Proposition 3.3 can be applied again and implies 
that there exists z” such that z' = Facing {!) o z" and -^Holds{Facing{l), z”). 
Combining this with (3.8) yields 

(3z") ( State{So) = At{l, 1) o Facing{l) o z" 

A -^Holds{At{lA),Facing{l) o z”) A Holds {Facing {!), z")) 

The conjunct ~^Holds{At{l, 1) , Facing{l) o z") can be simplified according to 
uniqueness-of-names along with decomposition and irreducibility, yielding 



(3z") ( State{So) = At{l, 1) o Facing{l) o z" 

A ~^Holds{At{l,T), z”) A Holds {Facing {!), z")) 



(3.9) 
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Having cast into a single equation all positive Holds statements of (3.1), next 
we incorporate the negative state knowledge. This will result in further con- 
straints on the state variable z” in (3.9). Consider, to begin with, the lit- 
eral ^ Holds {Occupied{l,l)T So). With regard to formula (3.9), it follows that 
Occupied{l, 1) does not hold in the term to the right of the equation sign. De- 
composition, irreducibility, and uniqueness-of-names imply that this is equiva- 
lent to Occupied{l,l) not occurring in z" , that is, 

(3z") ( State(So) = At{l, 1) o Facing{l) o z" 

A -^Holds{At{lA)^z”) A ^ Holds {Facing {l),z”) 

A -^Holds{Occupied{lA), z")) 

Adding the other pieces of information in (3.1) in a similar fashion yields 

(3z") ( State{So) = At{l, 1) o Facing{l) o z" 

A ~^Holds{At{l,l)T z”) A ^ Holds {Facing {l),z") 

A -^Holds{Occupied{lA)^z”) (3.10) 

A . . . /\ ^Holds{Occupied{4:,5), z") 

A (Vx, y) ^Holds{Cleaned{x, y), z ") ) 

This incomplete state specification is logically equivalent to (3.1) under the 
foundational axioms and the axiom of uniqueness-of-names for the fluents. The 
advantage of the modified representation is to provide a descriptive specification 
of State{So), which can be readily inserted into an update equation. Prior to 
this, the underlying domain constraints should be incorporated into the equa- 
tional state specification, too, in order to be able to deduce as much as possible 
about the updated state. Combining (3.10) with the constraints of the cleaning 
robot world, (3.2), results in further constraints on the state variable z”\ 

(3z") ( State{So) = At{l, 1) o Facing{l) o z" 

A Holds { O ccupied {1, 1), z”) 

A ...A ~^Holds{Occupied{4:,5), z") 

A iy x,y) ^ Holds {Cleaned {x,y), z") 

A (Vx, y) -nHolds{At{x, y),z") . . 

A {'id) -'Holds {Facing {d), z”) ' ' ' 

A {'ix) ^Holds{Occupied{x,0), z") 

A {'ix) ^Holds{Occupied{x,6), z") 

A I'iy) ^Holds{Occupied{0,y), z”) 

A I'^y) ^ Holds {Occupied {6, y), z")) 

Thereby, the two conjuncts ^Holds{At{l,l), z”) and ^Holds{Facing{l), z”) 
of formula (3.10) became redundant in the light of the additional constraints 
{'ix,y) ^Holds{At{x,y), z") and {'id) ^ Holds {Facing {d), z"). □ 

The general form of an incomplete equational state specification, suitable 
for insertion into an update equation, is (3z) {State{s) = toz A <&). Combining 
this with an equation State{Do{a, s)) = State{s) — d~ + deriving from a 
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state update axiom yields 

(3z) ( State{Do{a, s)) = t o z — A ) 

This formula provides a specification of the successor situation. If possible, 
rewriting steps may be applied to simplify the term to the right of the equation 
sign. In particular, negative effects may be “cancelled out” as in special fluent 
calculus with the help of the cancellation law (Proposition 1.7, page 10). 

Example 2 (continued) The three actions of the cleaning robot shall be 
represented by the following constants: 

Go : ACTION go forward to adjacent square 
Turn : ACTION turn clockwise by 90° 

Clean : ACTION empty waste bin at current location 

Let the precondition axioms be 

Poss{Go,z) = {\/d,x,y) { Holds{At{x,y), z) A 

Holds{Facing{d), z) D 

(3x', y') Adjacent{x, y, d, x' , y')) (3.12) 

Poss{Turn,z) = T 
Poss{Clean, z) = T 

in conjunction with the auxiliary axiom 

Adjacent{x, y,d,x' ,y') = 1 < d < 4 A 1 < x, x', y,y' < 5 A 

[d=l A x' = x A y' = y + lV 
d=2Ax' = x+ lAy' = y\/ (3.13) 

d=3Ax' = xAj/' = i/— IV 
d = 4 A x' = x — 1 A y' = y] 

That is to say, going forward requires the robot not to face the wall of the 
building while making a quarter turn clockwise and emptying a waste bin is 
always possible. The state update axioms for the three actions are as follows: 

Poss{Go, s) D 

(3d, X, y, x', y') ( Holds{At{x, y), s) A H olds (Facing (d) , s) A 
Adjacent(x, y, d, x', y') A 

State(Do(Go, s)) = State(s) — At(x, y) + At(x' , y ') ) 

Pass (Turn, s) D 

(3d) (Holds (Facing (d), s) A (3-14) 

State(Do(Turn, s)) = 

State(s) — Facing(d) + Facing (d mod I + 1) 

Pass (Clean, s) D 

(3x, y) (Holds(At(x, y),s) A 

State(Do(Clean, s)) = State(s) + Cleaned(x,y)) 
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Put in words, going forward brings the robot to the adjacent location. Turning 
has the effect that the cleanbot faces the direction which is next on the compass. 
Cleaning causes the waste bin in the current square to be cleaned. 

Recall the reformulation (3.11) of our sample initial state specification, and 
consider the action Go. From the first precondition axiom in (3.12) it follows 
that Poss{Go,Sq) since square (1,2) is adjacent to the initial position (1,1) 
in the direction the robot faces initially, viz. d = 1. Thus the state update 
axiom in (3.14) for Go implies 

State{Do{Go, Sq)) = State(So) - At{l, 1) + At{l, 2) 

Combining this update equation with formula (3.11), it follows that there ex- 
ists z” such that 

State{Do{Go, Sq)) = At{l, 1) o Facing{l) o z" — At{l, 1) -I- At(l, 2) 

along with all sub-formulas in (3.11) which constrain state variable z" . Macro 
expansion implies the existence of a state z such that ^Holds{At{l, 1), z) and 

z o At{l, 1) = At{l, 1) o Facing{l) o z" A State{Do{Go, Sq)) = z o At{l, 2) 

The equation to the left allows the application of the cancellation law thanks to 
^Holds{At{l,l),z”) according to (3.11), and due to the foundational axioms 
of irreducibility and decomposition along with uniqueness-of-names. It follows 
that z = Facing (1) o z" . Hence, along with all constraints on z" known from 
axiom (3.11) we obtain 

(3z") ( State{Do{Go, Sq)) = At{l, 2) o Facing{l) o z" 

A ~^Holds{Occupied{l,l), z”) 

A ...A ~^Holds{Occupied{4:,5), z'') 

A iy x,y) ^ Holds [Gleaned {x,y), z") 

A {yx,y)^Holds[At[x,y),z'') 

A yd) ^Holds[Facing[d), z") 

A yx) ^Holds[Occupied[x,0), z") 

A yx) ^Holds[Occupied[x,6), z") 

A yy) -'Holds [Occupied [0,y), z”) 

A yy) ^Holds[Occupied[6,y), z")) 

Thus we have derived an incomplete equational state specification of the suc- 
cessor situation. □ 

In general, an incomplete state may not entail whether a fluent to be added 
or removed currently holds or does not. In this case, partial information on this 
fluent entailed by the state may no longer be entailed by the updated state. 
Consider, for example, the partial state specification 



(3z) [State[s) = F[y) o z A [Holds[F[A)^ z) V Holds[F[B), z )\ ) (3.15) 
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along with the update equation 

State{Do{a, s)) = State(s) — F{A) 

With the usual inference scheme it follows that 

(3z) ( State{Do{a, s)) = {F{y) o z) - F{A) , , 

A [Holds{F{A),z)V Holds{F{B),z)]) ^ ’ 

This formula does not entail whether F(A) holds in the state it is subtracted 
from. By macro expansion of ” it follows that Estate U {(3.16)} implies 
-^Holds{F{A),Do{a,s)). But it cannot be decided whether F{y) holds in 
State{Do{a, s)) or whether F{B) does, since Estate U {(3.16)} entails 

[y = A D ~^Holds{F{y), Do{a, s)) ] A 
[y A D Holds{F{y)^ Do{a, .s))] A 
[^Holds{F{B), z) D ^Holds{F{B)^ Do{a, s))] A 
[Holds{F{B), z) D Holds{F{B), Do{a, s))] 

Hence, (3.16) just entails that 

(3z) {State{Do{a, s)) = z A ^Holds{F{A), z)) 

so that partial knowledge in (3.15) concerning the fluent that has been removed 
is not implied by the resulting state specification. 

Partial information is lost in the same way when adding a fluent wrt. an 
incomplete state specification which does not entail the status of this fluent. 
Consider, for example, the specification 

(3z) ( State{s) = z 

A [Holds{F{x),z)\J Holds{F{A),z)] (3-17) 

A -^Holds{F{B), z)) 

along with the update equation 

State{Do{a, s)) = State{s) + ^( 2 ;) 

With the usual inference scheme it follows that 

(3z) ( State{Do{a, s)) = z + F[x) 

A [Holds{F{x),z)\J Holds{F{A),z)] (3.18) 

A Holds {F (B), z)) 

This formula does not entail whether F{A) holds, nor whether F[B) does, 
since Estate U {(3.18)} implies 

[ Holds{F{A), z) V X = A D Holds{F{A), Do{a, s)) ] A 
[^Holds{F{A)^ z) A X ^ A D ^Holds{F{A)^Do{a,s))] A 
[x = B Zi Holds{F{B),Do{a,s))] A 
[x ^ B Z) ~^Holds{F{B), Do{a, s))] 
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Hence, (3.18) just entails that 

(3z) {State{Do{a, s)) = F{x) o z A Holds {F (x) , z)) 

so that partial knowledge in (3.17) concerning the fluent that has been added 
is not implied by the resulting state specification. 

The fundamental Theorem 1.14 of Chapter 1, which says that state update 
axioms solve the frame problem, carries over to general fluent calculus as it 
makes no assumption about states being completely specified. Hence, all parts 
of a state specification are still entailed wrt. the updated state if they are not 
potentially affected by the update. 

While states may be infinite in general fluent calculus, state update axioms 
are still restricted to finite positive and negative effects by definition. It is not too 
difficult to generalize the notion of an update equation to infinite collections of 
effects (see Exercise 3.6). However, this necessitates the intensional description 
of resulting states, which would spoil the computational merits of the extensional 
definition. For the moment, therefore, agent programs rely on the assumption 
that actions always have a bounded number of effects. Later, in Chapter 9, we 
will consider a different way of generalizing the concept of a state update axiom 
by which an unbounded number of effects can be specified in an extensional 
manner. 



3.3 Bibliographical Notes 

Reasoning about actions in the presence of incomplete states is intimately con- 
nected to the frame problem [McCarthy and Hayes, 1969]. The early approach 
of [Green, 1969] has simply circumvented this problem by introducing additional 
axioms for all non-effects of actions. Tracing back to a proposal of [Sandewall, 
1972], the search for more concise solutions to the frame problem has stim- 
ulated the development of nonmonotonic logics [Bobrow, 1980]. A promi- 
nent approach based on circumscription [McCarthy, 1986] has proved to be 
erroneous by an infamous counterexample known as the “Yale Shooting sce- 
nario” [Hanks and McDermott, 1987]. This has led to a variety of refined at- 
tempts to solve the frame problem in situation calculus, such as [Shoham, 1988; 
Kautz, 1986; Lifschitz, 1987; Baker, 1991; Turner, 1997; Kartha and Lifschitz, 
1995]. Some of these have turned out to cause other problems [Kartha, 1994; 
Stein and Morgenstern, 1994]. These difficulties with nonmonotonic approaches 
have inspired the search for concise solutions to the frame problem in classical 
logic [Kowalski, 1979; Haas, 1987; Schubert, 1990; Elkan, 1992], culminating in 
the concept of successor state axioms [Reiter, 1991]. State update axioms 
in fluent calculus can be viewed as a development of successor state axioms by 
introducing the notion of a state into situation calculus [Thielscher, 1999]. 

As an alternative to the situation-based representation of actions, the event 
calculus [Kowalski and Sergot, 1986] uses time points as a fundamental sort, 
which defines a linear time structure. Originally formulated as a logic program, 
a more general variant appeals to circumscription as a solution to the frame 
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problem [Shanahan, 1995]. Differences and common grounds of situation calcu- 
lus and event calculus have been extensively studied, e.g., [Kowalski and Sadri, 
1994; Miller and Shanahan, 1994]. 

Formal methods for assessing the range of applicability of action formalisms 
aim at comparing different approaches to the frame problem [Lifschitz, 1987; 
Lin and Shoham, 1991]. The two most prominent frameworks today originate 
in the articles [Sandewall, 1993a] and, introducing the action description 
language, [Gelfond and Lifschitz, 1993]. The former is topic of the textbook 
[Sandewall, 1994], in which a number of nonmonotonic solutions to the frame 
problem have been formally assessed. This method itself has led to an extensive 
action representation formalism [Doherty et al., 1998] based on a linear time 
structure as in event calculus. The action description language, on the other 
hand, has been used, for example, in [Kartha, 1993] to prove correctness of 
nonmonotonic action formalisms for well-defined problem classes. The language 
itself has been developed into various directions [Baral, 1995; Giunchiglia et al., 
1997; Kakas and Miller, 1997b] and has led to successful logic programming- 
based systems [McGain and Turner, 1998; Kakas et al., 2001]. 

Mainly around the time where many attempts to solve the frame problem 
have proved erroneous, the problem has been subject to philosophical reflec- 
tions [Dennet, 1984; Pylyshyn, 1987] questioning the feasibility of Artificial In- 
telligence in general. 
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3.4 Exercises 

3.1. Find a model for E^state U {(3z, f){\/z') z' z - /}. 

Hint : Construct a model for sort state which includes some but not all 
infinite sets of fluents. 

3.2. Suppose the cleanbot is initially at square (1,1) while it is unknown 
whether it faces north or west, that is, 

Holds{At{l,l), Sq) a [Holds{Facing{l), Sq) \/ Holds{Facing(4:), Sq)] 
Use the domain axioms for the cleaning robot to prove that 

(a) the robot does not face east initially; 

(b) the robot faces north or east after turning; 

(c) action Go is possible after turning; 

(d) if the robot is in square ( 1 , 2 ) after turning and going forward, then 
it faces north. 

3.3. Use the axiom of state existence to prove that 

(a) there is a state z in which the cleanbot is at square ( 1 , 1 ) facing 
north and in which all instances of Cleaned{x,y) hold; 

(b) for any state z\ there is a state Z 2 which is as zi except that no 
instance of Cleaned{x,y) holds. 

Show that if z\ in Exercise (b) is substituted by z of Exercise (a), then 
Z 2 of Exercise (b) satisfies Egtate \= Z 2 = At{lA) ° P<^cing{l). 

3.4. Prove that the axiomatization of the cleaning robot domain is determin- 
istic. 

3.5. (a) Define a macro “ 0 ” which generalizes ” to the subtraction of 

infinite states. 

(b) Let z\ be the state which consists of all instances of fluent G{n) 
where n e IN, and let Z 2 be the state which consists of all instances 
of fluent G{n) where n is an even natural number. Show that if 
zs = ziQ Z 2 , then Z 3 consists of all instances of G{n) where n is 
an odd number. 

(c) Prove that “ 0 ” is indeed a generalization of that is, 

Z2 = Zi - {fi O . . . O fn) = Z2 = ZiO{fiO...O /„) 

(d) Prove that foundational axioms Estate entail 



(Vzi, Z2)(3z3) Z3 = ZiQ Z2 
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3.6. (a) Use the previous exercise to define the macro for a generalized state 

equation Z 2 = {z\ Q z~) ® Z+ for arbitrary states 

(b) Generalize Theorem 1.14 to “0,0” and prove it. 

(c) Consider the fluent File{dir., file) with the intended meaning that 
file is in directory dir. Use “0, 0” to define an update axiom for the 
action Move*{dir\, dir 2 ) which moves all files from directory dir\ 
to dir 2 . Show that after Move*{“/tmp/” , “home/”) all files that 
were initially in /imp/ are in home/ and that directory /imp/ is 
empty. 

3.7. Design the environment of an agent to buy books at a given selection of 
Internet stores. 

(a) Consider the fiuents Price (&oofc, store, price), indicating that store 
has book in stock and sells it at price; GiftCertificate{store, price), 
indicating that the agent possesses electronic gift certificates for store 
of total value price; and, finally, InCart{hook, store), indicating that 
the agent has put book into its virtual cart at store . Define suitable 
domain constraints for these fiuents. 

(b) Define precondition axioms and state update axioms for the actions 
Add {book, store) and Pemore(6oofc, store), for adding and removing, 
respectively, book from the cart at store . 

(c) Define the auxiliary predicate Total{price, s) such that price is the 
total amount to be paid for the contents of the carts of all shops in 
the state of situation s. 

(d) Axiomatize the following information concerning the initial situa- 
tion: No store sells Bi = (“Gottlob Frege” , “Begriffsschrift” ) ; Store 
nile.com sells book B 2 = (“Jonathan Franzen” , “The Corrections”) 
at $20.95 while brownwell.com sells this book at $18.20; the agent 
has a gift certificate of $5 for nile.com; and all carts are empty. 
Show that no ground situation a = Do{[ai, . . . , an], Sq) exists such 
that (3x) Holds{InCart{Bi,x),a) is entailed. Prove that a ground 
situation a exists which satisfies 



Holds{InCart{B 2 ),a) A [Total{p,a) Dp<$16] 



Chapter 4 



General FLUX 



In this chapter, we use the ideas of general fluent calculus to extend the spe- 
cial FLUX system. The goal is to provide agents with the general ability to 
set up and maintain an incomplete world model and to reason about sensor 
information. To this end, we will present 

4.1. an encoding of incomplete states using constraints in logic programs; 

4.2. a method for handling these constraints; 

4.3. a proof that this method is correct wrt. the foundational axioms of fluent 
calculus; 

4.4. an extension of the FLUX kernel for the update of incomplete states. 

4.1 Incomplete FLUX States 

General FLUX is based on a particular technique for the encoding of incomplete 
state specifications. Incomplete collections of fluents are modeled by lists with 
a variable tail. This is a very natural way of representing compound states in 
fluent calculus which contain a state variable. For example, the term At{l, 1) o 
Facing(l) o z is encoded by this list: 

[ At (1,1), Facing (1) \ z] 

Incomplete lists cannot be subjected to the principle of negation-as-failure 
as employed in special FLUX for the evaluation of negated Holds expressions. 
This is because an assertion like, for example, 

~^Holds{Occupied{l, 1), [At{l, 1), Facing{l) \ z]) 

would simply fail: The corresponding affirmative Holds statement can always 
be satisfied via substituting the tail variable, z, by a list starting with the fluent 
in question. General FLUX, therefore, appeals to the paradigm of constraint 
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logic programming (CLP, for short). Certain atoms, the so-called constraints, 
which can occur in clause bodies and queries, may be partially evaluated only 
and kept in store for later consideration. 



Constraints add a great deal of flexibility and efficiency to logic programs. A 
standard constraint system is FD (for: finite domains), which includes arith- 
metic constraints over integers. Based on the standard functions -I-, — , and 
* , constraints are built using the usual equality, inequality, and ordering predi- 
cates. Range constraints of the form x :: [m..n] define an interval of natural 
numbers for a variable. All of these constraints can be logically combined us- 
ing conjunction and disjunction, respectively. Constraints may occur in the 
body of program clauses as well as in queries. In the course of a derivation, 
the encountered constraints are put into a constraint store. The contents 
of the store is constantly evaluated in the background, and a derivation fails 
as soon as an inconsistency among the constraints is detected. If a derivation 
succeeds with one or more constraints still unsolved, then these are part of the 
computed answer. For example, the query x :: [1..4], x>2*y,y>l results 
in the answer {y/1} along with the unsolved constraint x :: [3.. 4]. If the 
sample query included, say, a: < 3, it would fail. 



FLUX employs two kinds of constraints. First, special state constraints 
encode both negative and disjunctive state knowledge by restricting the tail 
variable of an incomplete list. Second, standard arithmetic constraints are 
used to define restrictions on unknown arguments of fluents. In so doing, FLUX 
assumes the arguments of fluents to be encoded by natural or rational numbers. 
This enables the use of a standard high-speed constraint solver. 

The available basic state constraints determine the set of state specifications 
that can be directly expressed in FLUX. 

Definition 4.1 A state formula 4>(2;) is FLUX-expressible if it is of the 
form 

(3f, z') {z = o . . . o ip^ o U A 4'i(f, z') A ... A 4'„(f, z')) (4.1) 

where m,n > 0, each ipi (1 < i < m) is a fluent with variables among x, and 
each 'i>j{x,z') (1 < j < n) is of one of the following forms: 

1. Holds {if, z'), where is a fluent with variables among x; 

2. (y y)^ Holds {ip, z'), where ip is a fluent with variables y; 

3. Holds{ip[,z') V ... V Holds{ip'f^,z'), where fc > 1 and ip{ (1 < j < fc) is a 
fluent with variables among x', 

4. quantifier-free arithmetic formula over integers with variables among x. 

□ 
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Thus, FLUX-expressible are state specifications which consist of existentially 
quantified positive and negated fluents, universally quantified negated fluents, 
positive disjunctions, and auxiliary axioms expressing properties of the argu- 
ments of fluents. For example, the fact that a fluent F has a unique value in a 
state z is FLUX-expressible by the formula 

(3x, z) {z = F(x) o z' A (Wy) ^ Holds (F{y), z)) (4-2) 

Likewise FLUX-expressible is the initial state specification in the cleaning robot 
domain, if we take the rewritten formula (3.11) on page 66. 

Universally quantified positive statements, like (Vn : N) Holds{F{n), z) , can- 
not be directly expressed with the state constraints considered in this book (but 
see Exercise 4.2 for how to encode universally quantified domain constraints by 
adding domain-specific constraints to a FLUX program). This ensures that 
only a finite number of fluents holds in a state. When designing a signature for 
FLUX agents, the programmer must therefore take care that the fluent func- 
tions are defined in such a way that all states can be encoded by finitely many 
positive fluents. In some cases this may be achieved by replacing a fluent F, 
of which infinitely many instances may be true in some state, by a new flu- 
ent F whose semantics is ~^F and, hence, of which infinitely many instances 
would be false in the same state. For example, it is not advisable to use a fluent 
NotInDirectory{x,y) to represent that a file x is not contained in a directory y; 
one should rather employ the semantic negation InDirectory{x,y). Further 
state formulas that are not FLUX-expressible according to Definition 4.1 are 
disjunctions with negated literals or exclusive disjunctions, but see Exercises 4.3 
and 4.5 for an extension of the expressiveness. 

FLUX-expressible formulas are encoded as so-called FLUX states, which 
consist of an incomplete list of fluents along with a set of constraints. The 
constraints encode the sub- formulas Tj of a state formula (4.1). An auxiliary 
constraint called DuplicateFree(z) is used to ensure that the state list z is free 
of multiple occurrences of fluents. 

Definition 4.2 Let (IF, o, 0) be a fluent calculus state signature. A FLUX 
state 4>(z) consists of a list 



2 = [/i, I z'] 

of pairwise different fluents (n > 0) along with the constraint DuplicateFree{z) 
and a finite number of constraints of the form 

1. NotHolds{f, z')-, 

2. NotHoldsAll{f, z'); 

3. OrffoMs([/{, z') where fc > 1; 



4. arithmetic constraints. 
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init(ZO) ZO = [at (1 , 1) ,f acing(l) I Z] , 
not_holds (occupiedCl , 1) , Z) , 

not_holds (occupied(2 , 1) , Z) , "/, hallway 

not_holds(occupied(4,5) , Z) , ’/. 

not_holds_all(cleaned(_,_) , Z) , 
consistent (ZO) . 

consistent (Z) 

holds(at(X,Y) ,Z,Z1) -> [X,Y]::1..5, not_holds_all (at (_,_), Zl) , 

holds(facing(D) ,Z,Z2) -> [D]::1..4, not_holds_all(facing(_) ,Z2) , 

not_holds_all(occupied(_,0) , Z) , 

not_holds_all(occupied(_,6) , Z) , 

not_holds_all(occupied(0,_) , Z) , 

not_holds_all(occupied(6,_) , Z) , 

duplicate_f ree (Z) . 



Figure 4.1: Sample state specification in general FLUX. 



If <j)( 2 ;) is a FLUX-expressible state formula (4.1), then by |4>(z)] we denote 
any FLUX state containing the same positive fluents and the respective set of 
constraints 1.-4. representing the sub- formulas 4'^. □ 

Again, we will distinguish between a FLUX-expressible formula and the actual 
FLUX state only when necessary. 

Example 2 (continued) Figure 4.1 shows the FLUX encoding of the initial 
state specification (3.11) (page 66) in the cleaning robot scenario. The auxiliary 
predicate Consistent{z) defines state z to be consistent in view of the under- 
lying domain constraints. A bit tricky is the encoding of the fact that there is 
always a unique instance of the two functional fluents. At and Facing. Inspired 
by formula (4.2), this is expressed via the ternary Holds{f,z,zi) known from 
special FLUX, where state z\ is z without fluent /. □ 



4.2 FLUX Constraint Solver* 

The introduction of constraints requires to extend the FLUX kernel by a con- 
straint solver. A standard constraint system for arithmetics is used to handle 
constraints on variable arguments of fluents. We follow the syntax of the FD- 
library for Eclipse, where the arithmetic predicates are preceded by the symbol 
to distinguish them from the standard Prolog operations. Conjunctions 
and disjunctions are denoted by “#/\” and “#\ /”, respectively. 

The second component of the constraint solver for general FLUX is a system 
of rules for handling the state constraints of Definition 4.2. 
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Constraint Handling Rules (CHRs, for short) are a general method of 
specifying, in a declarative way, rules for processing constraints. Consider 
an arbitrary signature for constraints which consists of predicate and func- 
tion symbols, variables, and the special constraint False. Then a Constraint 
Handling Rule is an expression of the form 

^ Gi,...,Gfe I (4.3) 

where 

• the head Hi, ... , is a sequence of constraints (m > 1); 

• the guard Gi, . . . , G*, and the body Ri, . . . , are queries {k, n > 0). 

An empty guard is omitted; the empty body is denoted by True . The declar- 
ative interpretation of a CHR of the form (4.3) is given by the formula 

(Vf) ( Gi A . . . A Gfe D [Ri A . . . A = (3y) (Ri A . . . A R„)] ) (4.4) 

where x are the variables in both guard and head and y are the variables 
which additionally occur in the body. Consider, e.g., a constraint Doublet{x) 
with the intended meaning that a; is a list of doublets, i.e., pairs of identical 
elements, as in [3,3,4, 4]. The following two CHRs can be used to handle 
instances of this constraint: 

Doublet {[xi,X 2 \y]) ^ - 1 X 1 = X 2 \ False (4.5) 

Doublet{[x\y]) y=[x\y'], Doublet{y') (4.6) 

Their interpretation is {ylxi,X 2 ,y) {x\ ^ X 2 D {Doublet{[xi,X 2 \y]) = T)) 
and (yx,y) {Doublet [[x\y]) = {3y') {y = [x\y'] A Doublet{y'))). 

The procedural interpretation of a CHR is given by an extension of the notion 
of a resolvent in a computation tree. If the head of a CHR can be matched 
against elements Ci, ... , Cm of the constraint store using a substitution 0 
and if the subsidiary computation tree for the guard Gi, . . . , Gfc is successful, 
then the constraints Gi, . . . , Cm are removed from the constraint store and the 
instantiated body (Ri, . . . , R„)0 of the CHR is added to the current query. If 
the special constraint False is thus introduced, then the entire derivation fails. 
Consider, e.g., the query Doublet{[3,x \ z\), then the guard in CHR (4.5) fails 
while CHR (4.6) is applicable and results in {a;/3} along with the pending 
constraint Doublet(z). The query RowWet([3, x, x, 4]), on the other hand, by 
CHR (4.6) and substitution {x/3} yields Doublet {[3, 4]), which in turn fails 
according to CHR (4.5). 

The notation Hi\H 2 <tA G | R abbreviates Hi,H 2 AA G\Hi,B. It is com- 
monly used to remove one or more constraints H 2 which are subsumed by 
constraints Hi. 
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not_holds (_ , [] ) <=> true. "/,! 

not_holds(F, [FlIZ]) <=> neq(F,Fl), not_holds(F,Z) . 7.2 

not_holds_all(_, [] ) <=> true. 7,3 

not_holds_all(F, [FlIZ]) <=> neq_all(F,Fl) , not_holds_all(F,Z) . 7.4 

not_holds_all(F,Z) \ not_holds(G,Z) <=> inst(G,F) I true. 7,5 

not_holds_all(F,Z) \ not_holds_all(G,Z) <=> inst(G,F) I true. 7,6 

duplicate_f ree ( [] ) <=> true. 7,7 

duplicate_f ree ( [F I Z] ) <=> not_holds(F,Z) , duplicate_free(Z) . 7.8 



neq(F,Fl) or_neq(exists ,F,F1) . 

neq_all(F,Fl) or_neq(forall,F,Fl) . 

or_neq(Q,Fx,Fy) functor (Fx,F,M) , functor (Fy,G,N) , 

( F=G, M=N -> Fx =.. [_|ArgX], Fy =.. [_|ArgY], 
or_neq(Q,ArgX,ArgY,D) , call(D) 

; true ) . 



or_neq(_, [],[], (0#\=0)) . 
or_neq(Q, [XlXl] , [YlYl] ,D) 
or_neq(Q,Xl,Yl,Dl) , 

( Q=forall, var(X) , \+ is_domain(X) 

-> ( binding (X, XI, Yl, YE) -> D=( (Y#\=YE)#\/D1) 

; D=D1 ) 

; D=((X#\=Y)#\/D1) ). 

bindingCX, [XllArgX] , [YllArgY] ,Y) X==X1 -> Y=Y1 

; bindingCX, ArgX,ArgY,Y) . 



Figure 4.2: CHRs for negation. 



4.2.1 Negation Constraints 

Figure 4.2 depicts the first part of the system of CHRs for FLUX, which han- 
dles the negation constraints as well as the auxiliary constraint on multiple 
occurrences. The topmost CHR defines NotHolds{f, z) to be true if z is the 
empty state, which is a direct encoding of the empty state axiom of fluent 
calculus. The second CHR propagates a constraint NotHolds{f, z) through a 
compound state, following the contraposition of the foundational axioms of de- 
composition and irreducibility. The auxiliary predicate Neq{f,fi) encodes the 
disequation f ^ fi and is defined by auxiliary clauses (see below) . In a similar 
fashion, the next two CHRs are used to evaluate NotHoldsAll{f, z), where the 
auxiliary predicate NeqAll{f, fi) encodes the disequation (Vaf) / ^ fi with 
X being the variables in /. By CHRs 5 and 6, a negation constraint of the 
form ~^Holds{g, z) or (\/x) ^ Holds {g, z) is resolved if it is subsumed by some 
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other negation constraint (Vy) -^Holds{f, z). The auxiliary predicate Inst{g, f) 
means that fluent g is an instance of fluent /. Finally, the two CHRs for the 
auxiliary constraint on multiple occurrences stipulate that DuplicateFree{z) is 
true for the empty list while compound lists are free of duplicates if the head 
does not occur in the tail and the tail itself is free of multiple occurrences. 

The auxiliary clauses in Figure 4.2 reduce the inequality of two fluents to 
an arithmetic constraint over the arguments. If the two fluent function sym- 
bols are unequal, then the clause for the ternary OrNeq succeeds, thus encod- 
ing that F{x) / G(y) under uniqueness-of-names. Otherwise, the resulting 
arithmetic constraint is a disjunction of argument-wise disequations. The base 
case of the empty disjunction is encoded as the unsatisflable constraint 0 7^ 0. 
The distinction between an existential disequation Neq{f,fi) and a universal 
NeqAU(f, fi) is made in the penultimate clause: In case of universal quantifi- 
cation, the pure variable arguments^ in / are discarded while possibly giving 
rise to dependencies among the arguments of the second fluent. These depen- 
dencies are inferred using the auxiliary predicate Binding{x, x\, yi, y), which is 
true if X reappears in xi and y occurs in yi at the same position as x does 
in Xi. For example, NeqAll{F{x,x,x),F{u,v,w)) reduces to the arithmetic 
constraint u ^ v \f v ^ w whereas NeqAll{F{x,2,y), F{u,v,w)) reduces to 
2 ^ V. In case of existentially quantified variables, all arguments participate 
in the resulting constraint, so that, e.g., Neq{F{x,2,y),F{u,v,w)) reduces to 
x^u\/2^v\/y=/:w. 

The following derivation illustrates these CHRs at work, using a query which 
combines the information that the cleaning robot is at a unique location (l,y) 
for y G {2,3} with the fact that the robot is not in the second row. Not 
surprisingly, the conclusion is that the robot is at location (1, 3). The elements 
in the constraint store are framed, and underlined are the atoms or constraints 
to which the next derivation step applies. The application of a CHR is indicated 
by its number according to Figure 4.2: 



y :: [2..3], z = [At{l^y)\ z\], NotHoldsAll{At{v,w), z\), 
NotHoldsAll{At{x, 2), z), DuplicateFree(z) 






y :: [2. .3] , z= [Ht(I, y) | 21], NotHoldsAll{At{v,w), zi), 



NotHoldsAll{At{x, 2),z), DuplicateFree{z) 






y :: [2. .3] , NotHoldsAll{At{v,w), Zi), 



NotHoldsAll{At{x,2),[At{l,y) l^i]), DuplicateFree{[At{l,y) |zi]) 






y :: [2. .3], NotHoldsAll{At{v,w), zi) , 

NotHoldsAU{At{x,2),[At(l,y) |zi]), DuplicateFree{[At{l,y) \zi\) 



^ A variable is pure if it does not satisfy the standard FD-predicate IsDomain(x) , which 
is true if x is a constrained variable. 
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y :: [2..3], NotHoldsAll{At{v,w), zi) 



NotHoldsAll{At{x,2),[At{l,y) \ Zi]) , DuplicateFree{[At{l,y) \ Zi]) 



y :: [2..3], NotHoldsAll(At{v,w), zi), NeqAll{At{x,2), At{l,y)) 



NotHoldsAll{At{x,2), zi) , DuplicateFree{[At{l,y) \ zi]) 



y [2. .3], NotHoldsAll{At{v,w), z\), 2 ^ y \J 0 7 ^ 0 


5 


NotHoldsAU(At(x, 2), zi) 


, DuplicateFree{[At{l,y)\ z\]) 


NotHoldsAll{At{v, w), z\) 


NotHoldsAll{At{x,2), z\) 


5 


DuplicateFree{[At{l,3) \ zi\) 




NotHoldsAll{At(v, w), Z\) 


, DuplicateFree{[At{\,3)\ 


^iD 


NotHoldsAll{At{v,w), z\), DuplicateFree{[At{l,3) | zi]) 



NotHoldsAll{At{v,w), zi), NotHolds{At{l,3), zi), DuplicateFree{z\) 



NotHoldsAll{At{v,w), zi), DuplicateFree(zi) 



Neither of the two resulting constraints can be resolved further. Hence, they are 
part of the computed answer, which otherwise consists of the substitution {y/3} 
determined by the arithmetic constraint solver from y :: [ 2 .. 3] and 2 ^ yWO ^ 0. 



4.2.2 Disjunction Constraints 

Figure 4.3 depicts the second part of the system of CHRs for FLUX, which 
handles disjunction constraints. The solver employs an extended notion of a 
disjunctive clause, where in addition to fluents each disjunction may include 
atoms of the form Eq{x,y). The meaning of such a general disjunctive con- 
straint OrHolds{[Si, . . . ,6k], z) is 

k 

y / r Holds{f, z) if 5i is fluent / , . 

^ \ X = y if is Eq(x,y) ^ 

This extension is needed for propagating disjunctions with variables through 
compound states. The constraint OrHolds{[F{x) , F{1)], [F{y)\z]), e.g., will be 
rewritten to OrHolds{[Eq{[l], [y]), F{1), Eq{[x], [y]) , F (x)] , z) . This is in accor- 
dance with the fact that Estate U UNA[F] entails 

Holds{F{x),F{y) o z) V Holds {F (1), F(y) o z) 

X = y V Holds{F{x), z) V 1 = j/ V Holds{F{l), z) 
which follows by the foundational axioms of irreducibility and decomposition. 
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or_holds ( [F] , Z) <=> F\=eq(_,_) I holds (F,Z). 7,9 

or_holds (V, Z) <=> \+(member (F, V) , F\=eq(_,_)) 7.10 

I or_and_eq(V,D) , call(D) . 

or_holds (V, [] ) <=> member (F,V,W) , F\=eq(_,_) I or_holds (W, [] ) . 7.11 

or_holds (V, Z) <=> member (eq(X,Y) ,V) , 7.12 

or_neq(exists,X,Y,D) , \+ call(D) I true. 

or_holds(V,Z) <=> member (eq(X.Y) ,V,W) , 7.13 

\+(and_eq(X,Y,D) , call(D)) I or_holds (W, Z) . 

not_holds(F,Z) \ or_holds(V,Z) <=> member (G,V,W) , F==G 7.14 

I or_holds(W,Z) . 

not_holds_all(F,Z) \ or_holds (V, Z) <=> member (G,V,W) , inst(G,F) 7.15 

I or_holds (W, Z) . 

or_holds (V, [F I Z] ) <=> or_holds (V, [] , [F I Z] ) . 7.16 

or_holds([Fl|V] ,W, [F|Z])<=> F1==F -> true ; 7.17 

F1\=F -> or_holds(V, [FlIW] , [F|Z]) ; 

Fl=. . LlArgX] , F=. . LlArgY] , 
or_holds(V, [eq(ArgX , ArgY) ,F1|W] , [F I Z] ) . 
or_holds ( [] ,W, [_ I Z] ) <=> or_holds(W,Z) . 7.18 



and_eq( [],[], (0#=0) ) . 

and_eq( [XlXl] , [YlYl] ,D) and_eq(Xl . Y1 ,D1) , D=( (X#=Y)#/\D1) . 
or_and_eq( [] , (0#\=0) ) . 

or_and_eq( [eq(X,Y) |Eq] , (Dl#\/D2)) and_eq(X,Y,Dl) , or_and_eq(Eq,D2) . 



Figure 4.3: CHRs for disjunction. 
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The first rule in Figure 4.3, CHR 9, handles a disjunction consisting of 
just a single fluent. Any such disjunction is simplified to a standard Holds 
statement. For example, the constraint OrHolds{[Occupied{l,y)], z) reduces 
to Holds{Occupied{l,y),z). 

Rule 10 simplifies a pure equality clause to a disjunctive arithmetic con- 
straint. For instance, the constraint OrHolds{[Eq{[x],[l]),Eq{[Z,y\,[x,‘S\)\,z) 
reduces to a; = lV (2 = a;Ay = 3). The next rule, CHR 11, is used to 
simplify a disjunctive clause in the presence of the empty state. For example, 
OrHolds{[Occupied{x,'i), Eq{[x\^\^])\,[\) reduces to OrHolds{[Eq{[x],[3])],[]). 
Auxiliary predicate Member{x,v) means that x is an element of v, and for 
Member[x,v,w) to be true, w must additionally equal v without x. 

CHRs 12 and 13 apply to disjunctions which include a decided equality. If 
the equality is true, then the entire disjunction is true, else if the equality is 
false, then it gets removed from the disjunction. For example, the constraint 
OrHolds{[Eq{[l,2],[l,2]), Facing{3)], z) reduces to True as 1 7 ^ 1 V 2 7 ^ 2 fails. 
Conversely, the constraint OrHolds{[Eq{[l,2],[x,3]), Eacing{x)], z) reduces to 
OrHolds{[Eacing{3)], z) as l = xA2 = 3 fails. 

Rules 14-15 model a unit resolution step, by which a positive atom in a 
disjunction is resolved in the presence of its negation. For example, given 
the negative constraint NotHoldsAU{Occupied{0,y), z), the disjunctive con- 
straint OrHolds{[Occupied{l,4),Occupied{0,3)],z) reduces to the disjunction 
OrHolds{[Occupied{l,4)], z) by CHR 15. 

The last group of CHRs, 16-18, encode the propagation of a disjunction 
through a compound state. Informally speaking, each element in the disjunct is 
matched against the head of the state, if possible, and the respective equational 
constraint is introduced into the disjunction. More precisely, with the help of 
the auxiliary ternary constraint OrHolds{v,w,[f\z]), a disjunction is divided 
into two parts. List v contains the fluents that have not yet been evaluated 
against the head / of the state list, while list w contains those fluents that 
have been evaluated. In the special case that the disjunction contains a fluent /i 
which is identical to the head / of the state list, the constraint necessarily holds 
and, hence, is resolved to True by CHR 17. Otherwise, any fluent /i in the 
disjunction which does not unify with / is propagated without inducing an 
equality. Any fluent fy which does unify with / extends the disjunction by 
the equality of the arguments of fi and /. For example, the constraint 

OrHolds{[F{x),F{l)], [F{y)\z]) 

mentioned earlier is re-written by CHR 16 to the ternary disjunction 
OrHolds{[F{x),F{l)],[],[F{y)\z]) 

Rule 17 then yields 

OrHolds{[F{l)], [Eq{[x], [y]),F{x)], [F{y)\z]) 

Applying the same rule again yields 

OrHolds{[], [Eq{[l], [y]), F{1), Eq{[x], [y]),F{x)], [F[y)\z]) 
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Now CHR 18 applies since all unifiable fluents have been evaluated, and the 
propagation is carried out by combining the equations with the remaining atoms 
of the disjunction, resulting in OrHolds{[Eq{[l], [y]), F(l), Eq{[x], [y]), F{x)],z). 

By appealing to the underlying arithmetic constraint solver, the auxiliary 
clauses in Figure 4.3 reduce equality of sequences of terms (and disjunctions of 
these equalities, respectively) to conjunctions of equations (disjunctions of these 
conjunctions, respectively). The clauses are similar to the encoding of disequa- 
tions in case of negation constraints. The base case of an empty conjunction 
is encoded by the tautological constraint 0 = 0 while an empty disjunction is 
encoded as before by the unsatisflable constraint 0^ 0. 

The following derivation illustrates the CHRs for disjunction at work, using 
a query which encodes the state knowledge that there is exactly one occupied 
office, that this office is in row 3 and in a column higher than 1, and that there 
is an occupied office in column 1 or 2. The constraint solver arrives at the 
conclusion that office (2, 3) is occupied. As before, the atoms and constraints 
are underlined which are resolved next, and the application of a CHR is indicated 
by its number:^ 

z = [Occupied{x,3) \ Zi], a; > 1, NotHoldsAll{Occupied{v,w), Zi), 
OrHolds{[Occupied{l, y), Occupied{2, y)], z) 
a; > 1, NotHoldsAll{Occupied{v,w), zi), 

OrHolds{[Occupied{l,y), Occupied(2,y)]^ [Occupied{x,3) |zi]) 

■H- a; > 1, NotHoldsAll{Occupied{v,'w),zi), 

OrHolds{[Occupied{l,y), Occupied {2, y)], [], [Occupied{x,'A) \ zi]) 

■H- a; > 1, NotHoldsAll{Occupied{v,w), zi), 

Or Holds {[Occupied (2, y)], [Eq{[l,y], [a;, 3]), Occupied{l,y)], 

[Occupied {x, 3) | Zi]) 

17 

<t=> a; > 1, NotHoldsAll{Occupied{v,w), zi), 

OrHolds{[], [Eq{[2, y], [a;, 3]), Occupied{2, y), 

i?y([l, y], [a;, 3]), Occupied{l, y)], [Occupied{x, 3) | zi]) 

H a; > 1, NotHoldsAU{Occupied{v,w), zi), 

OrHolds{[Eq{[2,y], [a;, 3]), Occupied{2,y), 

Eq{[l,y], [x,3]), Occupied{l,y)],zi) 

a; > 1, NotHoldsAll{Occupied{v,w), zi), 

QrHolds{[Eq{[2,y], [x, 3]), Ag([l, y], [a;, 3]), Occupied{l,y)], Zi) 



^ Since all but the very first step take place inside of the constraint store, we refrain from 
making explicit the steps by which the constraints enter the store, and we omit the frames 
around the elements of the constraint store. 
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X > I, NotHoldsAll{Occupied{v,w),Zi), 

OrHolds{[Eq{[2,y], [x,3]), Eq{[l,y], [a:,3])],Zi) 

IS- a: > 1 , NotHoldsAll{Occupied{v,w), Zi), 

{2 = X A y = 'S A 0 = 0) V (l = ccAy = 3A0 = 0) V 0^0 

NotHoldsAll{Occupied{v,w), z\) 

The computed answer substitution {x/2,y/3} is determined by the arithmetic 
constraint solver in the last step. 

The FLUX constraint solver endows agents with the general ability to log- 
ically combine and evaluate information about states. This enables agents to 
infer implicit knowledge from different observations, which means they are ca- 
pable of reasoning about sensor information and incomplete states. 

Example 2 (continued) The clause in Figure 4.4 encodes the specification 
of the light sensor in the world of the cleaning robot. Given the initial state 
specification of Figure 4.1, the FLUX constraint solver allows the cleanbot to 
infer implicit knowledge that follows from the perception of light at certain 
locations. The following two queries illustrate that the robot is able to evaluate 
given information about where and where not it would sense light: 

?- init(ZO), light(l,2,false,Z0) , light (1,3, true, ZO) . 

ZO = [at (1 , 1) , f acing(l) I Z] 

Constraints : 

not_holds (occupiedCl , 3) , Z) 

or_holds( [occupied(2 , 3) , occupiedCl ,4)] , Z) 



Put in words, no light in square (1,2) but in square (1,3) implies that of- 
fice (1,3) is not occupied but either office (2,3) or office (1,4) is so (or both). 
If the robot is additionally told that there is no light at location (2,2), then 
it concludes that office (2,3) cannot be occupied and, hence, that office (1,4) 
must be so: 

?- init(ZO), light(l,2,false,Z0) , light (1,3, true, ZO) , 
light(2,2,false,Z0) . 

ZO = [at(l,D, facing(l) , occupiedCl ,4) I _Z] 



Constraints : 

not_holds (occupiedCl , 3) , Z) 
not_holds (occupied(2 , 3) , Z) 



□ 
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light (X, Y, Percept, Z) 

X_east#=X+l, X_west#=X-l, Y_north#=Y+l , Y_south#=Y-l , 

( Percept=f alse , 

not_holds(occupied(X,Y) , Z) , 
not_holds(occupied(X_east,Y) , Z) , 
not_holds(occupied(X_west,Y) , Z) , 
not_holds(occupied(X,Y_north) , Z) , 
not_holds(occupied(X,Y_south) , Z) ; 

Percept=true, 

or_holds( [occupied(X,Y) , 

occupied (X_east ,Y) , occupied (X,Y_north) , 
occupied(X_west , Y) , occupied(X, Y_south)] , Z) ). 

Figure 4.4: A clause defining the functionality of the light sensor of the cleaning 
robot. 



4.3 Correctness of the Constraint Solver* 

The FLUX constraint solver can be proved correct under the semantics of fluent 
calculus. Thanks to their declarative nature, verification of Constraint Handling 
Rules against an axiomatic theory is not difflcult. 

4.3.1 Negation Handling 

Let us begin with the auxiliary clauses of Figure 4.2, which define inequality 
of two fluents in terms of an arithmetic constraint over the arguments. This 
is justified by the assumption of uniqueness-of-names for fluents. The solver 
distinguishes two cases, depending on whether the variables in the first fluent 
are existentially or universally quantified. In view of the latter, we introduce the 
notion of a schematic fluent / = h(x, r) with x being the variable arguments 
in / and r the ground arguments. The following observation implies that the 
clauses defining Neq and NeqAll, respectively, are correct. 

Observation 4.3 Consider a fluent ealeulus signature with a set IF of func- 
tions into sort FLUENT. Let fi=g{ri,...,rm) and f = h{t\, . . . ,tn) be two 
fluents and f '2 = g{xi , . . . , Xk, rk+i , . ■ • , r^) a schematic fluent. Furthermore, 
let NeqifiJ) = /i 7 ^ / and NeqAll{f 2 , f) = (Vxi , . . . , Xk) f 2 7 ^ f , then 

t- if 9 h, then UNA[F] |= Neq{f\,f) and UNA[F] ^ NeqAU(f 2 , f) ; 

2. if g = h, then m = n, and UNA[F] entails 

Neq{fi,f) = Ti 7 ^ V . . . V V 0 7 ^ 0 

NeqAll{f 2 ,f) = [V t, tj] V rfc+i tk+i V . . . V V 0 7 ^ 0 
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Based on this observation, the CHRs for negation constraints, 1-4, are justified 
by the following proposition. 

Proposition 4.4 Estate entails 

1. ~^Holds{f,%); and 

2. ~^Holds{f,fioz) = f ^ fi A^Holds{f,z). 

Likewise, if f = g(x,r) is a schematic fluent, then Estate entails 

3. i\/x) ^Holds{f,ttt); and 

4- {\/x)^Holds{f,fio z) = (Vf)/ 7 ^/i A (\/x) ^ Holds {f,z). 

Proof: 

1. Follows by the empty state axiom. 

2. We prove that Holds{f,f\ o z) = / = /i V Holds{f, z): 

“=>” : Follows by the foundational axioms of decomposition and irreducibil- 
ity. 

If / = /i, then f\o z = f o z, hence Holds{f,f\ o z). Likewise, if 
Holds{f, z), then z = f o z' for some z' , hence f\o z = f\o f o z' , 
hence Holds{f, fi o z). 

The proof of the second part is similar. ■ 

Rules 5-6 of the FLUX solver, by which subsumed negation constraints 
are resolved, are justified since (\/x) ^Holds{f, z) implies ~^Holds{g,z) for a 
schematic fluent / = h{x,f) and a fluent g such that fO = g for some 9. 
Likewise, fix)^Holds{f,z) implies fi if) ^ Holds {g, z) for any two schematic 
fluents f = hi{x,r) and g = h 2 {y,t) such that fO = g for some 9. 

Finally, CHRs 7-8 for the auxiliary constraint on multiple occurrences are 
correct since the empty list contains no duplicates while a non-empty list con- 
tains no duplicates iff the head does not occur in the tail and the tail itself is 
free of duplicates. 

4.3.2 Disjunction Handling 

Moving on to part two of the FLUX constraint solver, let us first consider 
the auxiliary clauses in Figure 4.3 which define equality of two sequences of 
terms, and disjunctions thereof. Their correctness is implied by the following 
observation. 

Observation 4.5 Let r = r\, . . . ,Vn and t = t\, . . . ,t„ be two sequences of 
terms of equal length n > 0, then 

r = t = n = ti A . . . A Tn = tn A 0 = 0 
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Furthermore, if ri,ti, . . . all are sequences of terms such that to > 0 

and for each 1 < i < to, r-i,ti are of equal length, then 

m 

\J[r, = ti\ = ri = fi V . . . V r„, = tm'dO^O 

i=l 



The first CHR in Figure 4.3 simplifies a singleton disjunction. Correctness 
of this rule follows by definition, because Or Holds {[ f ], z) means Holds{f,z). 
CHR 10, by which a pure equational disjunction is reduced to an arithmetic con- 
straint, is correct according to Observation 4.5. CHR 11, by which a disjunction 
is simplified in the presence of the empty state, is justified by the empty state 
axiom, which entails 

[HoWs(/, 0) V 4'] = 4' 

for any formula 4'. 

Rules 12 and 13 are justified by Observation 4.5 along with the fact that for 
any formula 4', 

r = t D [(r = rv4') = T] 
f^ t D [(r = fv4') = 4r ] 

Rules 14-15, which model unit resolution steps, are justified since for any for- 
mula 4r, 



[HoWs(/, z) V 4'] A ~^Holds{f,z) = ~^Holds{f,z) A 4' 
and, given that fi0 = f 2 for some 6, 

[HoMs(/ 2 , z) V 4'] A (yx) ^Holds{fi, z) = (yx) ^Holds{fi, z) A 4' 
where x are the variables of fi . 

Finally, consider CHRs 16-18, by which a disjunction is propagated through 
a compound state. Given that the semantics of the ternary OrHoldsfj, 6, [/ | z;]) 
is OrHolds{j,[f \ z]) V OrHolds{S, z), the rules are justified by the following 
proposition. 

Proposition 4.6 Consider a fluent calculus signature with a set T of func- 
tions into sort FLUENT. Foundational axioms Egtate uniqueness- of -names 

UNA[!F] entail each of the following : 

A 4- = [4'VVti^d; 

2 . [Ho/ds(/,/oz)v4'i] V4-2 = T; 

d- fi y f ^ { [Holds{fi,f o z)W 4'i] V 4»2 = 4»i V [Holds{fi,z) V 4»2] ); 

4 . [HoZds(F(af), F(j7) o z) V 4'i] V 4^2 = 4ri V [x = y V HoWs (F(a;), z) V 4 ^ 2 ] ; 

5- [VLi^^ V ^] = T. 
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holds (F, [F|_]) . 

holds(F,Z) :-nonvar(Z), Z=[F1|Z1], F\==F1, holds(F,Zl). 
holds (F. [F|Z] ,Z) . 

holdsCF.Z, [FlIZp]) Z=[F1|Z1], F\==F1, holds (F,Z1 , Zp) . 



Figure 4.5: The definition of Holds in general FLUX. 



Proof: Claims 1 and 5 are obvious. Claim 2 follows by the definition of 

Holds. Claims 3 and 4 follow from the foundational axioms of decomposition 
and irreducibility along with UNA[!F]. m 

Correctness of each single CHR implies that the constraint system for FLUX 
is correct, provided that the underlying arithmetic solver is correct. 

Theorem 4.7 Let P be the program of Figure 4.2 and 4-3 along with a 
eorrect arithmetic constraint solver. Consider a fluent calculus signature with a 
set T of functions into sort fluent, and let |4>(z)] be a FLUX state. If the 
derivation tree for PU{^ |4)(z)]} fails, then Estate^ ^(z) |= _L. 

Proof: Each CHR being an equivalence transformation on 4)(z), the claim 

follows by the correctness of the underlying arithmetic constraint solver. ■ 

4.4 Updating Incomplete FLUX States 

The constraint solver allows FLUX agents to reason about static incomplete 
state information. We now turn to the problem of reasoning about the effects of 
actions, that is, the computation of removal and addition of fluents to incomplete 
states. Generalizing the FLUX kernel of Chapter 2, Figure 4.5 depicts extended 
definitions for the Holds predicate. The only difference to special FLUX of 
Figure 2.2 (page 28) is the atom Nonvar(z) occurring in the recursive clause. 
The reason for this addition is that otherwise a query Holds (p, z) with vari- 
able z would admit infinitely many answers of the form z/[fi, . . . , fn,<p\ z'] 
(n > 0) with variables fi. Under the semantics of fluent calculus, these substi- 
tutions are all subsumed by the answer zj\g} \ z'\ obtained by just applying the 
non-recursive clause. 

Partially specified states may give rise to more than one way in which a 
fluent is derived to hold in a state. As an example, consider the derivation tree 
depicted in Figure 4.6. The leaves are labeled with the respective computed 
answer substitution along with the remaining elements of the constraint store. 

The following result generalizes Theorem 2.2 to the correctness of the clauses 
in Figure 4.5 in the presence of incomplete states. 

Theorem 4.8 Let P be the program of Figure 4-5 along with the FLUX 
constraint solver. Consider a fluent calculus state signature with fluent functions 
T , and let (p be a fluent and |4>(z)] a FLUX state. 
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2 = [F{x),F{y) I 2 i], DuplicateFree{z), Holds{F{A), z) 



Holds{F{A),[F{x),F{y)\z,]) 




{x/A} 



success 



Holds{F{A),[F{y)\z^\) 



NotHolds{F{A),zi) 
NotHolds{F{y), 21 ) 

A Ay 




success 



Holds{F{A),zi) 



{y/A} 



DuplicateFree{zi ) 



NotHolds{F{x), 2 i) 
NotHolds{F{A),zi) 
X A A 



success 



{2i/[F(A)|2']} 



DuplicateFree(zi) 



NotHolds{F{x),z') 
NotHolds{F{y), z') 
NotHolds{F{A),z') 



X Ay, X A A, y A A 



DuplicateFree{z') 



Figure 4.6: A derivation tree with three successful branches, each of which de- 
termines a different possibility for fluent F{A) to hold in the partially specified 
state F(j/) \zi]. The resulting negation constraints are all derived from 

the initial constraint DuplicateFree{z). 



1. If PU{<— \^{z)\, Holds {ip, z)} succeeds with answer 6, then 



If P U l^{z)J, Holds {ip, z)} fails, then 

Estate UNA[P]U{<^{z)}U{Holds{ip,z)} h -L 

Proof: Let z = [/i, | z'] be the state equation in |<I)( 2 )], then the 

foundational axioms of decomposition and irreducibility imply 



Sstate u {4>(z)} h Holds{ip, z) = \J [ifi = /;] V {3z") z' = ipo 2 " (4.8) 



1. If P U {<— l^{z)J, Holds{ip, z)} succeeds with answer 6, then ip9 = fiO 
for some 1 < i < n, or z'6 = [v? | -z"] for some z" . In either case the 
claim follows by (4.8). 

2. If PU \^{z)\,Holds{ip,z)} fails, then from the correctness of the 
constraint solver according to Theorem 4.7 it follows that 

'^state U UNA[T] U {4>(z)} 0{ip = fi\ 1= T 



state 1= ^{z)9 D Holds{ip, z)9 



n 
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for all z = 1, . . . , n, and also 

Estate u UNA[E] U {<^{z){z’ /[ip I z”]}} h ^ 

The claim follows by (4.8). ■ 

This theorem implies that there is a derivation for Holds {p, z) under FLUX 
state |4>(2;)] whenever there exists a state z satisfying the state specification 
4>(z) A Holds{p, z). Hence, the program can be used to verify whether it can be 
consistently asserted that some fluent p holds in a state. This is not the same 
as to say that p necessarily holds. For instance, a simple query like Holds [F, z) 
admits the computed answer {z/[F \ z']}. But the opposite is satisfiable, too: 
The query NotHolds{F, z) succeeds immediately. With general FLUX, it is 
therefore inappropriate to use Holds (or NotHolds) for conditioning in agent 
programs. Controlling the behavior of an agent in the presence of incomplete 
information requires a notion of knowing that a fluent does or does not hold. 
This will be the subject of the upcoming Chapter 5. For the moment, we note 
that Holds or NotHolds statements should be used in agent programs only to 
assert state information, as in the initial state specification of Figure 4.1 or the 
definition of light perception in Figure 4.4. 

Updating a partially specified state by a possibly non-ground fluent requires 
an extension of the definition of update as used in special FLUX. If the in- 
complete state entails the status of a fluent p, then the result of removing or 
adding p is inferred as before. On the other hand, in Chapter 3 we have seen 
that if the status of the fluent is not entailed by the state specification <l)(z) at 
hand, then partial knowledge of p in 4>(z) does not transfer to the resulting 
state. The loss of information when updating a state with a fluent whose sta- 
tus is unknown, is reflected in the extended definition of removal and addition 
of fluents in general FLUX; see Figure 4.7. The recursive clause for Minus 
defines z — f as before in case {4>(z)} U {Holds{f, z)} is unsatisfiable or in 
case {^{z)} Li {^Holds{f,z)} is unsatisfiable. Otherwise, i.e., if the status of / 
wrt. 4>(z) is unknown, then all partial information about / in 4>(z) is cancelled 
prior to asserting that / does not hold in the resulting state. Similarly, the 
recursive clause for Plus defines z + f as before in case {^{z)}U {Holds{f, z)} 
is unsatisfiable or in case {^{z)} Li {^Holds{f, z)} is unsatisfiable. Otherwise, 
if the status of / wrt. 4>(z) is unknown, then all partial information about / 
in 4)(z) is cancelled prior to adding / to the state and asserting that / does 
not hold in the tail. 

The definition of cancellation of a fluent / is given in Figure 4.8. In the base 
case, all negative and disjunctive state information affected by / is cancelled 
via the constraint Cancel[f,z). The latter is resolved itself by the auxiliary 
constraint Cancelled{f,z), indicating the termination of the cancellation pro- 
cedure. In the recursive clause for Cancel{f,z,zi), each atomic, positive state 
information that unifies with / is cancelled. 

This completes the kernel of general FLUX. To summarize, let Pkemei be the 
program of Figure 4.5, 4.7, and 4.8 along with the FLUX constraint solver. Much 
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holds (F, [F|Z] ,Z) . 
holds(F,Z, [FlIZp]) 

nonvar(Z), Z=[F1|Z1], F\==F1, holds(F,Zl,Zp) . 

minus (Z, [] ,Z) . 
minus (Z, [F I Fs] , Zp) 

( \+ not_holds(F,Z) -> holds (F,Z,Z1) ; 

\+ holds (F,Z) -> Z1 = Z ; 

cancel(F,Z,Zl) , not_holds(F,Zl) ), 
minus (Z1 ,Fs ,Zp) . 

plus(Z, [] ,Z) . 
plus(Z, [F|Fs] ,Zp) 

( \+ holds (F,Z) -> Z1=[F|Z] ; 

\+ not_holds(F,Z) -> Z1=Z ; 

cancel (F, Z, Z2) , Z1=[F|Z2], not_holds(F,Z2) ), 
plus (Z1 ,Fs , Zp) . 

update(Zl jThetaP ,ThetaN,Z2) 

minus (Z1 ,ThetaN,Z) , plus(Z,ThetaP,Z2) . 



Figure 4.7: FLUX clauses for updating incomplete states. 



like in case of special FLUX, soundness of the definition of update in general 
FLUX is established by a series of results concerning the auxiliary predicates. 
We begin with the auxiliary ternary Holds predicate. 

Lemma 4.9 Consider a fluent calculus state signature with fluent functions 
T, and let (p he a fluent and |4>(2;)] a FLUX state. 

1. If Ffcernei U Holds{p , z , zf)} succccds with answer 6, then 

"Estate ^ ^{z)9 D Holds{(f, z)6 f\{z — if = z\)9 

2. If Pfcernei u {^{z)}, Holds{(fi, z, Zi)} fails, then 

Estate UNA[E]U{<^{z)}U{Holds{p,z)} h -L 

Proof: Let z = [/i, . . . , /„ | z'] be the state equation in |4)(z)], then the 

foundational axioms of decomposition and irreducibility imply 

n 

P'state u {4>(2)} h Holds{(fi, z) = \/[(p = fi] V (3z") z' = f o z" (4.9) 

i=l 

1. If Pfcernei U Holds{(f, z, Zi)} succceds with answer 0, then ei- 

ther of two cases applies. 
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cancel (F,Z1,Z2) 

var(Zl) -> cancel(F,Zl) , cancelled(F,Zl) , Z2=Z1 ; 

Z1 = [G|Z] -> ( F\=G -> cancel (F, Z, Z3) , Z2=[G|Z3] 

; cancel (F,Z,Z2) ) ; 

Z1 = [] -> Z2 = [] . 

cancel (F,Z) \ not_holds(G,Z) <=> \+ F\=G I true, 

cancel (F,Z) \ not_holds_all(G,Z) <=> \+ F\=G I true. 

cancel(F,Z) \ or_holds(V,Z) <=> member (G,V) , \+ F\=G I true, 

cancel (F,Z), cancelled(F,Z) <=> true. 

Figure 4.8: Auxiliary clauses and CHRs for cancelling partial information about 
a fluent. 



(a) There exists some i = such that (f9 = fiO and where 

z\0 = [/i, . . . z']9. Thus, {ziO(p = z)9, and from 

DuplicateFree{z) in |4>(2:)] it follows that ~^Holds{ip9, z\9) . This 
and (4.9) imply the claim. 

(b) For some z" we have that z'9 = [tp \ z"] and z\9 = [/i, ...,/„ | z"]9. 
Thus, (zi o (^ = z)0, and from DuplicateFree(z) in |4>(z)] it follows 
that ^Holds{p9^z”) and ^Holds{p9, fi o . . . o . This and (4.9) 
imply the claim. 

2. Similar to the proof of Theorem 4.8. ■ 

Correctness of the definition for predicate Minus in case a fluent with un- 
known status is removed follows from the fundamental theorem of fluent calcu- 
lus: Following Theorem 1.14 (page 18), properties which are unaffected by an 
update continue to hold in the updated state. This applies to all parts of a state 
specification which are not cancelled. 

Lemma 4.10 Consider a fluent calculus state signature with fluent functions 
T , and let 'd~ he a finite state and |4>(2;)] a FLUX state, then for any 
the query PkemeiU>{^ [‘&(^)1j Afmits(z, z')} succeeds with answer 9 such 
that 

Estate u UNA[T] h Hz)0 D{z-d- = z')9 

Proof: Let I'd ] = [/i,...,/„], then Estate, h ^ = f\ ° fi ° ° fn 

according to the foundational axioms of associativity and commutativity. The 
proof is by induction on n. 

If n = 0, then = 0 and |i?“] = []. Hence, by the non-recursive clause 
the query Fkemei^ Minus [z,'d~ , z')} succeeds with answer 9 such 
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that z0 = z'd, which proves the claim since z — % = z' is equivalent to z = z' 
by definition. 

In case n > 0, we distinguish three cases. 

1. If Pkernei U \^{z)\, NotHolds{f\, z)} fails, then by correctness of the 

constraint solver (Theorem 4.7), Estate^ {^{z)}y4 {^Holds{fi,z)} ^ _L. 
Hence, Estate H Holds{fi, z) . From Lemma 4.9 it follows that 

Pkernei U {<— l^{z)J, Holds{fi, z, Zi)} succeeds with answer substitution 
9 such that Estate and ^{z)6 entail {z — ip = Zi)9. 

2. If Pkernei U l^{z)j , Holds{fi , z)} fails, then by correctness of the con- 
straint solver (Theorem 4.7), Estate'E{^{z)}U{Holds{fi, z)} ^ T. Hence, 
Estate \= ‘^(-j) 3 -rHolds{fi,z). The macro definition for ” implies that 
z — fi = Z\ is equivalent to Z\ = z. 

3. Consider any fluent ip occurring in the state equation of |4>(z)] such that 
Estate u UNA[T] h <7’ / /i- Then Estate u {4>(2:)} h Holds{ip,z) and, 
hence. Estate U {4>(z)} U UNA[P] \= z - fi = zi D Holds {(p, zi) . 

Consider any constraint NotHolds{<p, z') occurring in |4>(z)] such that 
Estate u UNA[T] ip ^ Ji. Then Estate u {4)(z)} ^ -rHolds{p,z) and, 
hence. Estate U {4>(z)} U UNA[P] \= z - fi = zi D ~rHolds{p, zi) by 
Theorem 1.14. 

Consider any constraint NotHoldsAll{p, z') occurring in |4>(z)] such that 
Estate U UNA[P] \= p ^ fi. Let x be the variables occurring in p, 
then Estate U {4>(2)} h (V^) -•Holdsip, z) and, hence. Estate U {4>(2)} U 
UNA[P] \= z — f I = Zi D (Vx) ^Holds{p, Zi) by Theorem 1.14. 

Consider any disjunction constraint OrHolds{[pi, . . . ,pk], z') occurring 
in |4>(z)] (where /c > 1) such that Estate U UNA[Jz] ^ ^ for all 

1 <i < k. Then Estate U{<i)(z)} h VLi Holds{pi,z) and, hence, SstateU 
{4)(z)} U UNA[P] \= z — f I = zi D Vfci Holds{pi, z\) by Theorem 1.14. 

The claim follows by the definition of the macro ” and the induction hypoth- 
esis. ■ 

In a similar fashion, the clauses for Plus define a sound update wrt. a 
collection of positive effects. 

Lemma 4.11 Consider a fluent ealeulus state signature with fluent funetions 
T, and let he a finite state and |4>(z)] a FLUX state, then for any I'd+J, 
the query Pkernei U {<— |4)(z)], PZws(z, l-d+J, z')} succeeds with answer 0 such 
that 

Estate U UNA[F] \= 4>(z)6» p{z + d+ = z')0 

Proof: Exercise 4.7. ■ 

The results culminate in the following theorem, which shows that the en- 
coding of update is sound. 
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adjacentCX, Y, D, XI, Yl) 

[X,Y,X1,Y1] : : 1. .5, 

D : : 1. .4. 

(D#=l) #/\ (X1#=X) #/\ (Y1#=Y+1) y. north 

#\/ 

(D#=2) #/\ (X1#=X+1) #/\ (Y1#=Y) y. east 

#\/ 

(D#=3) #/\ (X1#=X) #/\ (Y1#=Y-1) y. south 

#\/ 

(D#=4) #/\ (X1#=X-1) #/\ (Y1#=Y). y. west 



Figure 4.9: Defining adjacency with the help of arithmetic constraints. 



Theorem 4.12 Consider a fluent calculus state signature with fluent func- 
tions T , and let and i)~ be finite states and |4)(zi)] a FLUX state, then 
for any |'d+] and I'd”], the query 

Pkernel U [4>(2:i) 1, C/pdate(zi, |i?+|, |d 1,Z2)} 

succeeds with answer 6 such that 

"Estate U UNA[E] h ^(zi)d D {z2 = zi - rT + d+)0 

Proof: The claim follows from Lemma 4.10 and 4.11 along with the macro 

definition for update equations. ■ 

Example 2 (continued) Figure 4.9 shows the encoding of the notion of a 
location being adjacent to another one in one of the four directions. Consider 
the following sample query, where the predicate Consistent (z) shall be defined 
as in Figure 4.1, encoding the domain constraints of the cleaning robot world: 

init(ZO) :- ZO = [at (X, Y) ,facing(l) I Z] , 

X : : 4. .5, Y : : 1. .3, 
consistent (ZO) . 



?- init (ZO) , 

holds(at(X,Y) ,Z0) , holds (facing(D) ,Z0) , 

adjacent(X,Y,D,Xl,Yl) , 

update (ZO, [at(Xl,Yl)] , [at(X,Y)] ,Z1) . 

Z1 = [at(Xl,Yl) ,facing(l) I Z] 



Constraints : 

X :: [4..5] 

XI :: [4.. 5] 
Y :: [1..3] 
Yl :: [2.. 4] 
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Although they are non-ground, the status of both the negative effect At{x,y) 
and the positive effect At(xi,yi) is known wrt. the specification of the ini- 
tial state: The former is true while the latter is false due to the constraint 
NotHoldsAU(At{v,w), z), which derives from the domain constraint that there 
be a unique location. 

As an example of an effect with unknown status, consider the state specifi- 
cation (3x,y) -'Holds (Cleaned{x,y), Zi) along with the update equation Z 2 = 
zi + Cleaned{l, 1). Macro expansion and decomposition plus irreducibility imply 
that Holds{Cleaned{x,y), Z 2 ) if x=lAy = l and Holds {Cleaned{x, y), Z 2 ) if 
X ^ ly y ^ 1. Hence, it cannot be decided whether Cleaned{x,y) is affected or 
not by adding Cleaned{l, 1). Consequently, FLUX gives the following answer: 

?- not_holds(cleaned(X,Y) ,Z1) , duplicate_free (Zl) , 
update (Zl , [cleaned(l , 1)] , [] ,Z2) . 

Z2 = [cleaned(l,D I Zl] 

Constraints : 

not_holds( cleaned (1 ,1) , Zl) 
duplicate_f ree (Zl) 

□ 

While update in general FLUX is sound, as shown with Theorem 4.12, it 
is incomplete in the sense that it may not entail all logical consequences of an 
update equation. Consider, for example, the state specification 

{3x,y) (Holds (F{x), Zl) A Holds(F(y), zi) A x ^ y) (4-10) 

along with the update equation Z 2 = z\ — F(7). Although it is not entailed 
that F(x) holds in Z 2 nor that F(y) holds (because both x = 7 and y = 7 
are consistent), it does follow that Holds(F(x), Z 2 ) V Holds(F(y), Z 2 ) (because 
X = y = 7 is inconsistent with x ^ y). FLUX can only infer a weaker state 
specification: 

?- Zl = [f(X),f(Y) I Z] , duplicate_free (Zl) , 
update(Zl, [] , [f (7)1 ,Z2) . 

Z2 = Z 

Constraints : 

not_holds (f (7) , Z) 

duplicate_f ree (Z) 

X#\=Y 

This incompleteness is not an obstacle towards sound agent programs, because 
a state specification entails any logical consequence which is derived from a 
possibly weaker specification. 
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4.5 Bibliographical Notes 

Constraint logic programming has emerged in the early 1980’s as a powerful 
combination of declarative programming with efficient algorithms for particular 
data structures. The survey article [Jaffar and Maher, 1994] provides a gentle in- 
troduction to the theory and gives an overview of a variety of constraint solvers. 
The theory and practice of arithmetic constraints have been discussed in [jaffar 
et ai, 1992]. Constraint Handling Rules have been introduced in [Friihwirth, 
1998]. The article [Holzbaur and Friihwirth, 2000] provides an overview of ap- 
plications which use systems of CHRs. The FLUX constraint solver has first 
been presented and analyzed in [Thielscher, 2002b]. 
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4.6 Exercises 

4.1. Formulate the following questions as FLUX queries and answer them using 
the encoding of the cleaning robot domain. 

(a) If light is perceived in square (3,3) but not in square (2,4), what 
can be said as to whether light would be sensed in square (2,3)? 

(b) Consider a state in which the robot is somewhere in the top row, 
in which all offices in the fourth row are occupied, and in which the 
robot perceives no light at its current location. Where is the robot? 

4.2. While universally quantified positive state knowledge cannot be directly 
expressed in FLUX, you can often encode it indirectly by your own Con- 
straint Handling Rules. 

(a) Consider this additional domain constraint for the world of the clean- 
ing robot: 

Holds{Cleaned{x,y), s) D l<x,j/<5 

Let CleanedConsistent{z) represent that state z is consistent wrt. 
the domain constraint just mentioned. Construct suitable CHRs for 
CleanedConsistent , by which, e.g., 

y :: [1..5], CleanedConsistent([At{l,y), Cleaned{l,y — 3) \ z]) 

is reduced to y :: [4. .5], CleanedConsistent{z) . 

(b) Recall from Chapter 1 these domain constraints of the mail delivery 
world: 

Holds{Empty{b),s) D -^Holds{Carries{b,r), s) A 1 < b < k 
Holds{Carries{b,r),s) D !<6<fcAl<r<n 
Holds{Carries{b,Ti), s) A Holds{Carries{b,r 2 ), s) D ri = T 2 

Write CHRs (for fc = 3 and n = 6) by which incomplete states can 
be verified against these constraints. 

(c) Let the fluents Carries{x) and InsideOf{x,y) denote, respectively, 
that a robot carries x and that x is inside of y. Write CHRs to 
verify that an incomplete state satisfies this domain constraint: 

Holds {InsideOf{x,y),s) D 

[Holds{Carries{x), s) = Holds{Carries{y).,s)] 

Hint: Any positive instance of InsideOf {x, y) should trigger an aux- 
iliary constraint stipulating that Carries (x,y) is symmetric. 

Prove correctness of the CHRs using their declarative interpretation. 

4.3. Extend the FLUX constraint solver by a set of Constraint Handling Rules 
for the constraint XorHolds{[fi, . . . , fk], z) with the intended meaning 
that one and only one of fi, ■ ■ ■ , fk holds in z (fc > 1). Prove that the 
rules are correct under the foundational axioms of fluent calculus. 
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4.4. Use the Xor-constraint of the previous exercise to formalize the Litmus 
test: Encode the state knowledge that a chemical solution is acidic just in 
case the Litmus strip is not red. What follows after adding that the strip 
is red, and what follows by adding that it is not red? 

4.5. Extend the FLUX constraint solver by a set of Constraint Handling Rules 
for the constraint AllHolds{f, [fi, . . . , fk], z) with the intended meaning 
that all instances of / but /i, . . . , /^ hold in z {k > 0). Prove that the 
rules are correct under the foundational axioms of fluent calculus. 

4.6. Solve Exercise 3.2 in FLUX: Encode the initial state zq of the cleaning 
robot to be in (1, 1) and to face either north or west. Use FLUX to show 
that 

(a) initially, the robot does not face east; 

(b) let Zi = Zq — Facing{d) + Facing {d mod 4+ 1) such that d satisfies 
Holds {Facing (d), Zq) , then the robot faces north or east in zi; 

(c) it cannot be that in zi of Exercise (b) there is no square adjacent 
to the location of the robot in the direction it faces; 

(d) let Z 2 = zi — At{x,y) + At{x',y') such that Holds{At{x,y),zi), 
Holds {Facing {d), Zi) , and Adjacent{x,y,d,x' ,y'), and suppose that 
Holds{At{l,2), Z 2 ), then the robot faces north in Z 2 . 

4.7. Prove Lemma 4.11. 

4.8. This exercise is concerned with a simple computer game known as the 
“Wumpus World” (see also Figure 4.10): An agent moves in a grid of 
cells, some of which contain, initially unknown to the agent, bottomless 
pits or gold, and there is one square which houses the hostile Wumpus. 
Its sensing capabilities allow the agent to perceive a breeze (a stench, 
respectively) if it is adjacent to a cell containing a pit (the Wumpus, 
respectively), and the agent notices a glitter in any cell containing gold. 
Consider the fluents At{x,y), Pit{x,y), Wumpus{x, y), smd Gold{x,y), 
denoting that in square (x, y) there is the agent, a pit, the Wumpus, and 
gold, respectively; the fluent Facing{d) as in the cleaning robot world; the 
fluent Alive denoting that the Wumpus is alive; and the fluent Has{x) 
for X G {Arrow, Gold} to denote that the agent is in possession of an 
arrow and gold, respectively. 

(a) Design suitable domain constraints for the Wumpus World. Encode 
these constraints in FLUX by a clause which defines the predicate 
Gonsistent{z) . 

(b) Define auxiliary predicates Breeze{x,y,v, z), Stench{x,y,v, z), and 
Glitter{x,y,v, z) such that v = True if the respective sensation is 
to be expected at square {x,y) instate z, and v = False otherwise. 



4.6. EXERCISES 



101 





Figure 4.10: A sample scenario in a Wumpus World where the 5x5 cave features 
three pits, the Wumpus in cell (3, 1), and gold in cell (4,4). In the right hand 
side are depicted the corresponding perceptions (breeze, stench, glitter) for each 
location. 



(c) Use the FLUX constraint solver to prove that if it is only a breeze that 
is perceived in cell (1,2) and only a stench in cell (2,1) (as depicted 
in the right hand side of Figure 4.10), then it follows that there is no 
gold in either of these squares, that there is a pit in cell (1, 3), and 
that the Wumpus hides in cell (3, 1). 



Chapter 5 



Knowledge Programming 



Robotic agents should always behave cautiously: No matter whether a certain 
action would in fact be possible and successful, an agent should not attempt 
the action unless it knows that the action can be performed and has the desired 
effect. Otherwise the agent program runs the risk to send a command to the 
physical apparatus which cannot be executed in the current state of affairs, or 
which has unintended or even dangerous effects. The programmer therefore has 
to take the subjective perspective of an agent. In this chapter, we show how this 
perspective can be integrated into fluent calculus and used in agent programs. 

The difference between the objective and the subjective view is most obvious 
when agents use their sensors. Sensing helps the agent with learning more about 
its environment. This has usually no effect on the external world itself while it 
does affect the subjective perspective. Hence, the full effect of actions involving 
sensing is not reflected by a mere state update. For instance, when the office 
cleaning robot moves to another square, then from the objective perspective 
it just changes its location. This is precisely what the state update axiom for 
action Go says. Whether or not the cleanbot senses light at the new place 
is irrelevant to the physical world, but it enhances the knowledge of the robot 
about the world. A suitable model of knowledge and sensing will allow us to 
write control programs for robotic agents which have to make decisions under 
uncertainty and which can use their sensors to acquire additional information. 
To this end, we develop 

5.1. a definition of what it means to know something under incomplete infor- 
mation; 

5.2. a method for FLUX agents to infer what they know; 

5.3. a method for FLUX agents to update their knowledge upon performing 
actions and receiving sensor input; 

5.4. a way of encoding domain descriptions in FLUX which deal with incom- 
plete knowledge; 
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Figure 5.1: A sample knowledge state in the cleaning robot domain: Possible 
are the states in which Facing{2) and either At(l,2) or At{2,2) hold. All 
other states are considered impossible. Hence, in this situation the robot knows 
it faces 2 and is either at location (1,2) or at location (2,2). 



5.5. an exemplary control program that guides our cleanbot safely through its 
environment while cleaning as many locations as (cognitively!) possible. 

5.1 Representing State Knowledge 

Incomplete state knowledge is represented in fluent calculus via the notion of 
possible states. A possible state is any world state in which the agent could 
be according to what it knows. The diversity of possible states is an indicator 
of how much information an agent has. In the ideal case of complete knowledge, 
the only possible state is the actual one. At the other end of the scale, a fully 
ignorant agent (say, a robot which has just been switched on without being told 
where it is) considers possible just any state. A typical partial knowledge state, 
however, is characterized by several possible and impossible states. An example 
is sketched in Figure 5.1. 

For the representation of possible states, the basic signature of fluent calculus 
is extended by the predicate KState{s,z) (for: knowledge state). The intended 
meaning is that — as far as the agent knows — z is a possible state in situation s. 

Definition 5.1 A tuple S U (KState) is a fiuent calculus signature for 
knowledge if 5 is a fluent calculus signature and 

KState : SIT x STATE 

A knowledge state for a situation a is a formula KState{a, z) = 4>(z) where 
4)(z) is a state formula. □ 

For example, suppose that initially the cleaning robot only knows that it faces 
east and is either at (1,2) or at (2,2) and that none of the waste bins has been 
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cleaned yet. This, then, is the knowledge state of the robot in situation Sg: 

KState{So,z) = H olds {Facing {2), z) A 

(dxo) {Holds{At{xo, 2), z) A [a:o = 1 V xo = 2]) A , . 

{\/u,v) ^Holds{Cleaned{u,v)^ z) A 
Consistent {z) 

where Consistent {z) stands for the domain constraints in the cleanbot world, 
that is, axioms (3.2) on page 61, with s being replaced hy z. In this way, 
we assume that the cleanbot is aware of all domain constraints: It considers 
possible only those states which are physically plausible. Hence, the robot shall 
always at the very least know that it is at a unique location facing a unique 
direction, etc. 

The new predicate enables us to define formally what it means for agents 
to know certain aspects of the state of their environment. Informally speak- 
ing, a state property is known just in case it is shared by all possible states. 
Equivalently, we can say that a property is known if the opposite is considered 
impossible. Conversely, unknown to the agent is everything which is false in at 
least one possible state. Knowledge state (5.1), for example, implies that the 
cleaning robot knows that it faces east and that it is not in a column higher 
than 2. On the other hand, the robot does not know that, say, room (5, 1) is 
not occupied, because one possible state is Facing{T) o (1, 2) o Occupied{5, 1) . 
Likewise, its precise location is unknown to the robot. Nonetheless, with a lit- 
tle logical reasoning the robot should be able to conclude that a Co action is 
possible in the initial situation: No matter which of At{l,2) or Tt(2, 2) holds, 
there is a square in direction 2 (i.e., east) from the current location. Hence, 
according to precondition axioms (3.12) on page 67, Poss{Co, z) holds in all 
possible states in So; formally, (Vz) {KState{So, z) D Poss{Co, z)) . 

The examples illustrate that it is natural to be interested in knowledge of 
complex matters, such as whether it is known that the cleaning robot faces a 
direction such that the adjacent square is inside of the boundaries. The following 
notion of a knowledge expression allows to form such compound properties of 
states. While they resemble genuine formulas, these expressions acquire meaning 
only via a macro for knowledge. This macro says that a knowledge expression is 
known in a situation just in case it holds in all possible states in that situation. 

Definition 5.2 A knowledge expression is composed of the standard log- 
ical connectives and atoms of the form 

1. /, where / fluent; 

2. Poss{a), where a action; 

3. atoms P{t) without terms of sort state or sit. 

Let 4> he a knowledge expression, then 

Knows{4>, s) =* (Vz) {KState{s, z) D HOLDS{(j), z)) 
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where HOLDS{(f>, z) is obtained by replacing in (f> each fluent / by Holds{f, z) 
and each Poss{a) by Poss{a,z). Furthermore, for an n-ary fluent function F, 
let X be a sequence of n pairwise different variables (n > 1) and let x\ be a 
non-empty subset thereof, then 

Knows Val {x i, F{x), s) = {3xi) Knows{{3x2) F{x),s) 

where X 2 are the variables in x but not in ali. □ 

As an example, (3x,y) {At{x,y) A x < 3) is a knowledge expression which is 
known wrt. knowledge state (5.1): 

Knows{{3x, y) {At{x, y) A x <3), Sq) 

= KState{So,z) D HOLDS{(3x,y) (At(x,y) A x < 3), z) 

= KState{So,z) D (3x,y) (Holds (At(x,y), z) A x < 3) 

<= KState(So,z) D (3xo) {Holds(At{xo, ‘2), z) A [xq = 1 V xq = 2]) 

The second macro, KnowsVal, is used to express that an agent knows a value 
of a fluent, that is, an instance for the given variables which is true in all possible 
states. For example, knowledge state (5.1) entails KnowsVal(d, Facing(d)^ Sg) 
since there exists d such that Facing (d) holds in all possible states (namely, 
d = 2). On the other hand, (5.1) entails that no value for At(x,y) is known. 
To prove this, consider the two possible states zi = At(l,2) o Facing(2) and 
Z 2 = At{2,2) o Facing (2). From UNA[At, Facing] along with decomposition 
and irreducibility it follows 

^ (3x, y) {Holds(At{x, y),zi) A Holds(At{x, y),Z 2 )) 

Hence, since KState(So, zi) A KState(So, Z 2 ), 

(3x,y)(\/z) (KState(So, z) D Holds(At(x,y), z)) (5.2) 

that is, ^KnowsVal((x,y), At(x,y), So). A partial value, however, is known for 
fluent At (x, y), namely, the y-coordinate: 

KnowsVal(y, At(x, y),So) 

= (3y) Knows((3x) At(x,y),So) 

=* (3y) (KState(So, z) D (3x) Holds (At (x,y), z)) 

KState(So,z) D (3xq) (Holds(At(xo,2), z) A [xq = 1 V Xq = 2]) 

When verifying knowledge, the choice of the right knowledge expression is 
crucial. For instance, knowing a value for a fluent is much stronger than merely 
knowing that the fluent has a value. With regard to knowledge state (5.1), for 
example, the knowledge expression (3x, y) At(x, y) is true in all possible states; 
hence. 



z)(3x,y) (KState(So, z) D Holds(At(x,y), z)) 



(5.3) 
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Contrasting this to formula (5.2) illustrates how important it is to choose 
the quantifier structure which correctly reflects the intended assertion. For- 
mula (5.3) can also be written as Knows{{3x,y) At{x,y), Sq), which represents 
knowledge de dicto (that is, “of a proposition”), as opposed to knowledge de re 
(that is, “of an object”) as in (3x,y) Knows{At{x,y), So) . It is not difficult to 
prove that the latter implies the former but not vice versa (see also Exercise 5.2). 

A universal property of knowledge is to be true. In fluent calculus, this is 
formally captured by a simple additional foundational axiom. 

Definition 5.3 The foundational axioms of fluent calculus for knowl- 
edge are Estate U Eknows where Estate is as in Definition 3.1 (page 63) and 
Eknows contains the single axiom 

(Vs) KState{s, State[s)) 



□ 

The following proposition says that under this axiomatic foundation everything 
an agent knows of a state does indeed hold in the actual state. Another property 
of the axiomatization of knowledge is that agents are logically omniscient. That 
is to say, agents know all deductive consequences of what they know, at least in 
theory. 

Proposition 5.4 Let cj) and ip be knowledge expressions, then foundational 
axioms Estate C Ef^jiQiss entail 

1. Knows{4>, s) D HOLDS {4>, State{s)) . 

2. Knows{4>, s) D Knows(ip,s), if HOLDS{4>, z) \= HOLDS{ip, z) . 

Proof: 

1. Suppose Knows{(p, s), then macro expansion implies HOLDS{(p, z) for 
all z such that KState{s,z)] hence, HOLDS {(p, State(s)) by Eknows- 

2. HOLDS{(p, z) \= HOLDSpip, z) implies HOLDS{cp,z) D HOLDS(ip,z) 

for all z such that KState{s, z); hence, by macro expansion it follows 
that Knows{(p,s) implies Knows{ip,s). ■ 

In practice, logical omniscience is not necessarily a desirable property, because 
universal inference capabilities are usually computationally infeasible. Due to 
the incompleteness of update in FLUX (cf. Section 4.4), FLUX agents are in 
general not logically omniscient. 



5.2 Inferring Knowledge in FLUX 

The concept of knowing a property of the state is essential for the evaluation 
of conditions in agent programs under incomplete information. It replaces the 
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simpler notion of a state property to hold, which we have used in special FLUX 
to condition the behavior of agents. 

By definition, a property is known just in case it is true in all possible 
states. From a computational perspective, it is of course impractical to evaluate 
a condition by literally checking every possible state, since there is usually quite 
a number, often even infinitely many of them. Fortunately, there is a feasible 
alternative. Instead of verifying that all states satisfy a property, we can just 
as well prove that the negation of the property is unsatisfiable under a given 
knowledge state. This approach is justified by the following theorem. 

Theorem 5.5 Let KState{a, z) = be a knowledge state and (j) be a 

knowledge expression, then 

{KState{a, z) = ^{z)} |= Knows{4>,(j) iff {^{z), HOLDS z)} |= _L 

Proof : 

{KState{a, z) = ^{z)} ^ Knows{(j),a) 
iff {KState{a, z) = d>(z)} ^ (Vz) {KState{a, z) D HOLDS{<f>, z)) 
iff ^ (Vz) ($(z) D HOLDS{ct>, z)) 
iff 1= ->{3z) {^{z) A HOLDS{-^(p, z)) 
iff mz), HOLDS i^<l),z)} h -L 



This fundamental result suggests a rather elegant way of encoding knowl- 
edge in FLUX. To begin with, if we want to verify knowledge in a situa- 
tion a with knowledge state KState{cr, z) = then we can just refer to 

the (incomplete) state specification d)(z). Furthermore, in order to verify that 
{d)(z), HOLDS {-'(j), z)} is unsatisfiable, negation-as-failure can be employed to 
prove that HOLDS z) cannot be consistently asserted under state specifi- 
cation d>(z). Figure 5.2 shows how this is realized in FLUX for the basic knowl- 
edge expressions / and ->/, where / is a fluent. More complex knowledge 
expressions, such as disjunctive knowledge, can be encoded in a similar fashion 
(see also Exercise 5.7). It is important to observe that both Knows{f, z) and 
KnowsNot{f, z) should only be used for ground fluents /. The auxiliary pred- 
icate KHolds{f, z) can be used instead in order to infer known instances of a 
fluent expression. 

Figure 5.2 includes a FLUX definition of macro KnowsVal. To begin with, 
the auxiliary predicate KHolds{f,z) maf c/ies fluent expression / against a flu- 
ent fi that occurs positively in z. Matching requires to test that fi is an in- 
stance of / before unifying the two terms. For the sake of efficiency, the further 
encoding is based on the extra-logical Prolog predicates Assert and Retraet 
used in conjunction with backtracking: The auxiliary, unary KnowsVal(x) is 
true just in case the current constraints entail a unique instance for the variables 
in X. To this end, Dom(x) infers a consistent instance for all variables in x 
with the help of the standard FD-predicates IsDomain(x) (which is true if x 
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knows (F, Z) \+ not_holds(F, Z) . 

knows_not(F, Z) : - \+ holds (F, Z) . 

knows_val(X, F, Z) k_holds(F, Z) , knows_val(X) . 

k_holds(F, Z) nonvar(Z) , Z=[F1|Z1], 

( instance (FI, F) , F=F1 ; k_holds(F, Zl) ). 

dynamic known_val/l. 

knows_val(X) dom(X) , ground (X) , ambiguous(X) -> false. 

knows_val(X) retract (known_val(X) ) . 

dom ( [] ) . 

dom([X|Xs]) dom(Xs) , ( is_domain(X) -> indomain (X) 

; true ) . 

ambiguous (X) retract (known_val(_) ) -> true 

assert (known_val(X) ) , false. 

Figure 5.2: Knowledge in FLUX. 



is a constrained variable) and Indomain(x) (which derives a consistent value 
for x). If X is not a constrained variable, it needs to be ground in order to 
have a unique value. The auxiliary predicate Ambiguous (x) fails upon asserting 
the first inferred instance for x. The predicate succeeds upon the second call, 
that is, after successfully backtracking over Dom(x), which indicates multiple 
possible values for x. 

Example 2 (continued) Recall from Figure 4.1 (page 78) the FLUX specifi- 
cation for the initial state in the cleaning robot scenario. Suppose, for the sake 
of argument, it is also given that there is no light perceived in square (1,2) but 
in square (1,3). Then we can use FLUX to show that the robot knows that 
room (1,3) is not occupied, while it does not know that office (1,4) is free, 
nor that it is not so: 

?- init(Z), light(l,2,false,Z) , 
light (1,3, true, Z) , 
knows_not (occupiedd ,3) ,Z) , 

\+ knows(occupied(l,4) ,Z) , 

\+ knows_not (occupiedd, 4) ,Z) . 



yes. 
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In terms of fluent calculus, let 

KState(So,z) = ^o(z) A ^Light(l,2, z) A Light(l,3, z) 

where ‘ho(-z) is the initial state description for the cleanbot, axiom (3.11) on 
page 66, with State(So) replaced by z. Then Knows(^Occupied(l,3), So) and 
~^Knows(Occupied(l,4), So) as well as ~^Knows(^Occupied(l,4), So)- 

As an example for the FLUX definition of knowing values for fluents, consider 
the FLUX encoding of knowledge state (5.1): 

init(ZO) ZO = [at(X,2) ,facing(2) I Z] , 

X#=l #\/ X#=2, 

rLOt_holds_all(cleaned(_,_) ,Z0) , 
consistent (ZO) . 



?- init (ZO) , 

knows_val( [D] ,facing(D) ,Z0) , 

\+ knows_val( [X,Y] ,at(X,Y) ,Z0) , 
knows_val ( [Y] , at (_ , Y) , ZO) . 

D = 2 
Y = 2 

While functional fluents like the position or orientation of the robot allow at 
most one value to be known in consistent knowledge states, other fluents may 
admit several known values for their arguments: 

init(ZO) :- ZO = [cleaned(l,2) ,clecUied(4,Y) I Z] , Y :: 2.. 4, 
consistent (ZO) . 

?- init(ZO), knows_val( [X] .cleaned (X,_) ,Z0) . 

X = 1 More? 

X = 4 



□ 

Let the FLUX kernel program Ptemei be extended by the clauses of Fig- 
ure 5.2. Soundness of the definition of knowledge in FLUX follows from the 
soundness of the underlying constraint system. Theorem 4.7, and Theorem 5.5. 

Theorem 5.6 Consider a FLUX state 4>(z), fluents ip and F(x), and a 
non-empty subset of x. Let S be Estate ^ {KState{a, z) = ^{z)} . 

1. If the derivation tree for Pkemei U {<— 4)(z), Ano?/;s(:/9, z)} is successful, 
then E ^ Knows{ip,a) . 

2. If the derivation tree for Pf^f,rneiU>{^ ^{z), KnowsNot{(p, z)} is successful, 
then E ^ Knows{-'ip,a) . 
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3. If 0 is computed answer to Pkemei U {<— ^{z),KnowsVal{xi,F{x),z)}, 
then 

(a) Yi\= KnowsVal{xi,F{x),a). 

(b) E \= Knows{(3x2) F{x)9,a). 

where X 2 are the variables in x besides X\. 

Proof: We prove each statement separately. 

1. Suppose Pkernei U {<— ^{z), Knows{(fi, z)} IS successful, then the query 
Pkernei U {<— ^{z) , NotHolds {(fi, z)} falls. From Theorem 4.7 it follows 
that Estate O {^{z),^Holds{if, z)} |= -L. Therefore, E ^ Knows{ip,a) 
according to Theorem 5.5. 

2. Suppose PkerneiEl{^ ^{z), KnowsNot{ip, z)} is successful, then the query 

Pkernei U ^{z) , Holds {(fi, z)} fails. From Theorem 4.7 it follows that 

EstateO{^{z), Holds (if, z)} ^ T. Therefore, E \= Knows{-^ip,a) accord- 
ing to Theorem 5.5. 

3. Suppose 9 is an answer to Pkemei U ^{z) , Holds{F{x) , z)} , and Xi9 

is ground. By Theorem 4.8, 

state ^ ^{z)9 D Holds{F{x), z)9 (5.4) 

Let z = [fi, ■ . . , fn \ z'] be the state equation in FLUX state 4>(z). As xi 
is non-empty and 0 is a combination of most general unifiers, groundness 
of x\9 implies that 9 does not affect [/i, ...,/„ | z'], so 4)(z)0 = 4)(z). 
Then (5.4) implies 

state ^ 4)(z) D Holds{F{x)9, z) 

Therefore, Sjtate U {‘h(-^)} is inconsistent with -rHolds{F{x)9, z). Hence, 
since Xi0 is ground, E ^ Knows{(3x2) F{x)9,a), which implies that 
E \= KnowsVal{xi, F{x),a). m 



5.3 Knowledge Update Axioms 

Knowledge of state properties is dynamic. Upon performing an action, agents 
need to update their knowledge according to the effect of the action and by 
incorporating acquired sensor information. Much like the update of the actual 
state is determined by the relation between some State{s) and the successor 
State{Do{a, s)) , knowledge update means to infer the knowledge state in the 
successor situation Do{a,s) relative to the knowledge state in situation s. To 
this end, the states which are possible after the performance of an action need 
to be related to the possible states prior to it. 

A schematic illustration of knowledge update is given in Figure 5.3. Be- 
ing uncertain about which state the cleanbot is in, the knowledge state after 
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Figure 5.3: Knowledge update: The set of possible states after the action is 
determined by the effects on the possible states in the original situation. 



performing the action Clean is obtained by updating each one of the initially 
possible states according to the effects of this action. Because the precise initial 
location of the robot is not known, we arrive at two kinds of possible states 
in the successor situation: The robot is either at square (1,2) and the waste 
bin there has been cleaned, or it is at square (2, 2) and the waste bin at this 
location has been cleaned. Other combinations are not possible, that is, the 
robot cannot be at (1,2) while having cleaned the bin at location (2,2) or 
vice versa. Moreover, unaffected knowledge, like Facing(2) , is still known in 
the resulting situation as it continues to hold in all possible states. 

If an action involves sensing, knowledge update has the additional effect 
that some states are rendered impossible. Thus the set of possible states 
shrinks according to the acquired knowledge. Figure 5.4 depicts an exam- 
ple of the update caused by a pure information gathering action, where the 
robot uses a locating sensor. Assuming that the robot is actually at loca- 
tion (1,2) in situation Si = Do{Clean, So) , only those possible states re- 
main which contain At{l,2). Incidentally, since all possible states z' satisfy 
Holds{Cleaned{l,2), z') , the robot also learns which of the waste bins it has 
cleaned. 

In fluent calculus, the effect of an action on the knowledge of the agent is 
formally specified by a so-called knowledge update axiom, which adds a cognitive 
effect to the physical effect as described by mere state update axioms. Cognitive 
and physical effect together determine the knowledge state KState{Do{a, a), z') 
relative to KState{a,z). 

Definition 5.7 Consider a fluent calculus signature with action functions A 
and a set of state update axioms for A. Consider an action A G A along 

with its state update axiom in 



Poss{A{x),s) D 'E[State{Do{A{x),s)), State{s)] 



(5.5) 



